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meee hiaence) orocessr, the 7ec1S$i90n makina process,s and the 
Meerati sna forces reauired to achieve a oarticular mission. 
Unfortunately, the majority of the resources spent in the 
imorovetment of these systems are exoended ie) either 
mma Hetter gata aathering, failsafe communicationss or 
Better weapon svstens. Miteeite —enOuUGht 15 Given to how the 
Meemsion naker orocesses the data in order to nake effective 
decisions. Several comcuterized decision aids have been 
pmeregrested to fil) this can. One of these aids, Onrerations 
marmeente!lliagence (NPINT) is evaluated to assess its utility 
Meme oncuteri7ed decision aid in C5 anoolications. OPINT is 
an one-line, interactive, realw-tine decision aia which 
assists decision makers by prescribing a straiahtforward 
Memeacive orocesdure for organizing and analyzing difficult 
Meersion oronliers. The resuits of the exreriment Shcw tnat 
Meee aids in the decision raking orocess, but has some 


mever= limitations as it currently exists. 
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Peete Or MECTSION 4N4LYSIS 


Maemo titres of crisi1S or wars there 18 orobabdiv no 
Meea of endeavor like military and strategic command and 
eontro! where Geeisueomses are  mede aquickly,s out under 
Mmemagictions of risk and extreme uncertainty. This 1S 
esnecially true since accurate information on all ascects of 
merecorem are almost always concealed from cecision makers. 
Every military force tries to conceal itself and maxes every 
attemot to deceive the orvosiny decision maker as to its 
intentions. Facts surrounding the tactical situation are 
Bmicayver esocecially those concerning the enemy. Tnese tyoes 
Of decisions, then, are difficult and frequently would seem 
to defy a systematic decision makina erocess (Derartment. of 
mime Army, 1980, o. 3-2). At the same time, there 18 no 
mma im which bad decisions have more tragic results. Tne 
mer oOr MakKINa errors Can arrow exoonentially mainly due to 
the corplex relationsnios and resultina chain reactions, or 
the errors could cancel themselves out and the costs not be 
immediately known (Turban and Meredith, 1977, o. 4). 

Consequently, wnile decision makers must recognize that 
Seer ly, rational decision makina orocedures are difficult, 
they must at the same time constantiv strive to base 
BeGsions on the most rational thoucht orocess cossible. To 
reduce error opronability, and for the sake of survival, 


Pecnsion maxers must bCecome more sophisticated (Turban and 





Meredith, 1977, On Se) Preveemusteleanrnm to utilize new 
tools and tecnniques that are heing develoned. ie Ome “—Could 
imagine a successful Someeconr weUttlizinmG equionent and 
memoceaures from the turn of the century. Yet, in Aecisi0)8)Hn 
maxing. woumecam st1!l tind Gecision mekers using the tools 
Bmoerecnniaques of that time. 

Bamercary ang strategic decisions are not only ‘nade 
mmerercOndi tions of uncertainty (where not al] facts: ore 
availiantle) and stress, they are also rarely considered 
Meee Ine ever changina situation brinas with it the 
requirement tO cont tnu0cus! y revise aporaisals, estimates, 
meme rnaos gecisions. Mino weeGecurs Simce Gecisicn makina 15 
based on the future but 1$ Geoendent on the past. For 
years, manaaers nave considered decision makina to he a oure 
Siereeor talent which 31S acquired over a long oeriod of time 
mmmpouahn experience oor trial ages [ert ot. it] has heen 
considereg an art Mecause © 46 wide variety of individual 
mes Gan OF USED 1M AaMMroCACHhINGa and successfully solving 
tne sane tvyde of moroolens. One wouls nornrally rase these 
Styies on creativityr judqment, Totuat 1} on, and experience 
meemer than some sort of systematic method (Turban and 
Memegith, 1977, cr. 5). 

Meecisivom makina as a discinline hag its origins in 
Mmeemetional analysis techniques beginning in sWorld war Ii 
map tliiams, 1978, o. 12). These techniques were tyoically 
acolied to special tvyvoes of clearcut, renetitive problems, 


meen as those of systematic search and resource allocation. 





Since the 1960's, however, a more aeneral technology 
eeeeemerged for imcosina loaqical structure on the reasoning 
mummmimcder!ies any snecific decision. [his Sec nme VoGy 1S 
meemom. analysis (S8arciay et alee 1977sr po. iv). Since 
Mmm, chere 3s been a najor effort by defense agencies to 
adoot this ScemPolodyetonrtnelc Gday=to=-cday decision making. 
Many have found it away to Make better, more defensiole 
decisions. 

Decision analysis 1S a Quantitative methoa which 
Semmrts ithe systematic evaluation of the costs or benefits 
meetin from courses of action that miaht be taken in a 
Seeersyon Oroblem (Barclay et ale, 19774 0o.f. vide The method 
includes the jiaentification of the alternative choices 
involved, the assiannent of values (costs/nenefits) for 
Beast ole outcomes, and the exoression of the orobanility = of 
Mmeooe OUtCOTesS occurring. Once this is done, the orobable 
aqain or loss associated with eacn alternative can he 
Mmetermined Ov Systematically combinina the nmrobahilities and 
values. 

Mregaay1tion to the primary cole of decision analysis as 
meee unod for the  loaical SoU@tomns yor comolex cCecis1o0n 
problems, it also has several additional advantages as well. 
The formal structure of decision analysis insures all the 
elements, their relationshios, and their associated weiahts 
have heen considered in the decision pronlem. The model of 
the decisien rrorlen can serve an iaimoortant role oan 


mer titatina communications between those Involved mn the 





@ecr+rsion orccess. NSO) it is very easy to identify the 
Breas of disaareement, their relative immpo9rtance, and if 
they actually have anv material wm@aet om the indicateno 
Pemeearonm. Finally, when chanyes occur in the problem, it 15s 
relatively easy to reenter the existing oroblem structure to 
meeamae Values or to add or remove oroblem dimensions as 
memmneea (fFarclay et ale, 1977, oe. vii). 

Tt should he emohasizved that in no sense does decision 


7 


mae | ¥S1S reolace decision makers or the roje of human 


judament ie IeECISION —Makina. Intuitive, iO ute) ts or 
judamental weowmaToneemmeciagd 1S, arter all, the mainstay of 
the exmerienced decision maker and for good reason. 
Intuitive unw3a1ided methods have tyoically served the 


decision nakxer well, and ne/she can reflect on some oersona! 
feeeary of DOroductive retiance on their developina intuition 
mepmkers, |19/c, 9. 191). Tnere 18 consideradle evidence 
to Show Pest Uimaided decision makina i$ reasonable, 
effective, and reliable (Peterson and S8each, LI GT too. 
29-46). However, what decision analysis does, 18 DOrovide an 
orderly ana more easily understood structure that helos to 
aggregate the wisdom of exnmerts on the many tooics that may 
be needed to nake a decisions and then suopvort the skilled 
decision maker by providing him with sound techniques to 
Ssumolement and ensure the internal eomaistency of 9n1s 
judqrent. 

Mamowex tecisiom oroblems are often difficult to 


resolve. This occurs for a number of reasons. Ontions are 


10 





pemea ways ciesarliv sefined. Any results which may be 
derived Peover em sce I=ct yon Of A Particular option may be 
meer y Urcertain. Qetueortcee ise orten gitficult Eo determine 
mewestive oreferences fae teem OOsSSih |e GCecision outcomes. 
when problems such as theSe do occur, the decision maxer 
normally takes steos to Structure the oroblem and reduce it 
Memes more exolicit form. MinbSeeyS Mexactly ~what decision 
ear ysis coes. 

Necision analysis builds uoon four nasic elements whicn 
Reem aerant im any decision orohlem (Harclay et al., 1977, 
Se meine use of these four elements allows for a smooth 
@umereedure In the resolution of comolex decisions. The four 


elenents ares: 


une A set of i wara! Courses Of action. You must 
have more tnan one alternative or there iS NO 
decision to be made. All nossible alternatives 
snould cre considered A eyo ult regard to 


Seesiicoe yt rty at this ooint. 


a. The vossiole consequences of each initial aGikis 
These must ne considered. what are the 
important thinas that can haeren that will make 
one act more valuable or worth more than another 
act? Relevant sequences of sunsequent events 


and follow7uo acts must be identified for each 


mage so) «6c tee 


Bes How attractive OG piMateract 1Vve 1S each 


IL 





Bemseaience “ot Aach SiGe How desiradle oor 
Undesirarle 1s one outcome comoared to others 
moon) Ty anes pest! t fCor the Same or anotner 


Mec? S10N.« 


ae How likely is it tnat a weerticulear act aie 
result in each of the consequences. Veaaes 
Preoojol ey, of UnMcettainty Can be measured as a 


mmeeacv lity troy 9 to | or in the form of odds. 


These four elements, as descrined, orovide away £6 
e6e5ni2e, Mant tt y+ itor acemtae bogical inolications of 
mmeeoecisicn. Vem onrinarsy vObplective 1S to provide a model 
os Perot Ooart (ali would ne nest) of the decision. The 
use cf the word model in this case neans to reoresent tae 
meenwsioOn 31n a auantifiable form, 

imams a Central oreceot of aecision analysis that all 
relevant considerations in a aecisicn Can be represented 
Memmi m og decision diacranr fBarciay et ale» 19777 =. Eun 
ies. 422615109 daisaaraz will Show everything a decision maker 
feels is relevant to the oroblem in auestion. A decision 
maeaoram consists essentially of a network of branches 
corresoonding to vossible sequences of actS and events, 
memmana cut from an oriain at the left tno a time horizon at 
Mmmemeriant. Acts are available choices. Events are nossible 
Beeurrences which are oartliy or comoletely outside the 
Meension naker's control, though the chance cf one of them 


hapnenina may he influenced by acts which were carrieo out 


lea 





earlier. 

mere 9eC1S10N dW1iSsqgrar crachic3liv distinauishes acts 
from events. Act for<s are reoresented hy sauares and event 
momeceoare reor=sented by circles as shown in Fiacure 1. 

These act PoOnmesmeama Event forks are then combinem to 
form a decision tree. An exanole of a tree 1§ Shown jin 
Prauire ce PwfpcmeOeGrecTom sitUation tavolves a comoany which 
must decide whether to hid on two eprojects, A aa eee The 
mersiom on oroject A must he made prior to the decision on 
es 

geo ceC1S102 diagram depicts all the oossirle acts and 
events and shows how these relate to each other in the 
Meersion situaticn. ‘hile the representation would help a 
Ae€cision maker to see at a alance his alternatives and 
Mmeemcrry those thinas that miaht affect any choice to be 
rade, ant does not yet answer the central auestion: whicn 
memomce should be made? That auestion can net be answered 
without corsicdierina tne value of the oossible outcomes and 
meer yk<eithood af occurrence of the events. iG Ole Cie et © 
mmomeenow this is incoroorated into the decision model, let's 
@emrm to the nitriding exanole. Let's say it has been 
meculated it will cost $10-,00 to orenmare sa bid for either 
Broject. MaeeGre wha on 41S wom, a aain of $50,000 wil) be 
realized, Becwecctunoewi!t a'so return a again of $50,000 if 
meen id 1S won but due to overheads wil!) only return a gain 
et mano +f bio A is also won. The aecision tree now 


meees like Figure 3. ines aumoers Shownm at the right side of 
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EtGURE 3: BEelstTON TREE WIThR VALUES ADDED 
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the diaarar are tne nath values. These values are the sum 
femmmerorecosts and gains alona each oath. 

Cw wo caTolete Peed agranm, we neeo to oortray tne 
Searee Of uncertainty about the events we cannot GOntCRO! + 
Sereemryeally whether 3 oid 18 won or lost. based Soon eps tela 
exoerience, it 1§$ Judaenr that Teme ten SOM PoOro lec t A 1S 
Sempre likely to be won or lost (orobability .5 wins .5 
hese). mm SN oOm, Resp rohebility of winning the bid on Bb 
is reduced to ess However, if 4 1s fost, the chances of 
mmr ss «INCcrease to ./, and if 4 1s not bid on, the 
Mmamamees for 8 are .60 to wine Adainag these values J}eaves our 
Seeemam !ookingo like Finaure 4. 

This Srackam Ssoould now be a virtually comolete 
meermsiation or mogel of the cercection of the decision raker 


9 f tne Ceacision waco ler. 


determine which 1s the best 


mavolves tne Galle lot tom 


Pxoected values, 


Popmeac nh 8ooG is leon oocti on. 


elie Uno t is left Now 31S to 


SO uit tO Fis iemmethod to do this 


Suewevonmtedn values, often called 


rHius technique 


1$ the sinclest and statistically most Strairantforward 
method that can ne used. Care must be taken however, since 
frequently there are decision circumstances wherein. an 
exoected value solution to a decision orohblem may not be an 


Bot ina | One®. 


wee are required and will 


Peterminina 


mea orocedire called folding 


Values 


In these cases, 


aevertue tor each act 


aeewectwoetTtutecd for each act and event 


alternative means of treating 


De escussed later. 


imera GOeC1S10M 15S done 


back the decision diagram. 


Pelpi levsehitala’ y lane! 
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memperesrignt-hand side of the diaaram. &£t each event CORK? 
the exoected value of that event 16 calculaten by 
emeemervinda the valtie of each oossible outcome by the 
Sgmerelritvy of the occiturrence of that outcome. Kt eacn act 
fork, the act with the hiahest value is the one which should 
Brvyve the best decision. The crocess of substitution is 
Seemed until the initial act fork is reached. Figure 9 
Memicts this nrocedure. Therefore the decision maxer would 
mmemeomoroject 4, then if he won tne bids not bid on Project 
Bs fee toast the B14 or A, he woulda bid on project &. 

emo tar in the discussion of decision analysis, the 
values of tne gains or losses were assigned in terms of only 
one measure, money. The outcones were then comovared, usING 
mmemeeexoected value, very easily Since it is easy for us to 
relate to money aS a value. HOwevers in most real life 
Pema 1OnS, esoecially those of the military, agains and 
losses are not measure inn Ome yr rather ele terrains 
equipment, sao at lessee mareven lives (Nilliams, 1978, o. 
Cc). These value dimensions are aQualitative rather than 
Mmumeieritative and are very difficult to measure. To further 
cloud the 1¢6suer, each Individual has his own personal 
subjective values for each of these. 

In addition, verv seldom are these value dimensions or 
attributes aiven 4nual weiaht in a decision. For example, 
©Gonsicer the selecticn of a radio set from several competing 
models. Pectin vehe am Incoortant consideration, Sut so are 


meyqght, ranqe, Gortan) itv, aad Seley onic ty. Just HOw 
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Mememrmmant 23ch 1$ reaguires cetermination. What 1S Imoortant 
memes = 8 aec)isi1on 1S whee a 9 Oo f these attrioutes is more 
Meet ant than the otners and how is this difference scaled. 
meemm ion GO the remainina attricutes fit on this scale. mor: 
MrseanrGce, i3re cost and reliability equally imreortant? If 
MOt, what exactly is the difference and how does the 
mecision maker make trade-offs between the two. 
Beme@ermrmsticns of this sort must ce mage for all attrinutes 
and somehow our nodel must te asnle to cComoare across these 
many attrioutes and aaaren3ste the results EC 1nAadicate the 
mest Solution. 

Proolems of this nature Fave sourred the develooment of 
Semeemeattrioute utility models (williams, POV Ey Os, ~cd)< 
These roagels nels Boy oO roviade a ee ee ve ranking for 
Betriroutes and also orovide a common auide for aagqregating 
the measures into a sinale inex of worth. T51S. somocess 
wAvolves the assianment of a measure of utility or merit to 
the attributes. 

Utility can be describet as a Subjective measure of 
mmm CF arctay et ales 1977, 3. 27). It 18 & personal 
value reflecting how you subjectively value Something. The 
meomycation of utility aS a measure Olt the value of an 
meeryoute or alternative was first oroposed by Von Neuman 
and Morgenstern. They suggested that eacn individual has 4 
measurable oreference among various choices available. The 
preference they called utility and is measured in arbitrary 


mmits eaweleog eat 1 !es (Von Neuman and “oraenstern, 1°44, oc. 


al 





Eat) Meplity 1S based on the conceot Eh at ineec 1S 10N 
nakina, 3 eerson will choose that alternative which 
emmnmrzes 11S ofr her expected utility . 

Popcimattrinute utility motels have heen used 
extensively in the systems acquisition role (Williams, 1978, 
Bie cao)» Tre procedures encourage discussion among the 
decision trakcer and his staff, No tlonger can cost be a 
memmnemmore inoortant than reliahnility. Peis) AOw, Cerhaos, 
Bemus ' 2S TOre imoortant. ince the values are assiaqned to 
the mnodel, and a rest decision deciaen uo0on, the model now 
allowS you to vary the relative weights to see which ones 
have the nost iicogttnon etme EfOoDpOsed Solution. This is 
called a sensitivity analysis and helos decision makers when 
mueyerare uncertain about the accuracy of their information. 

Decision analysis anpolies savesian techniques to olace 
mevasrie on information that reduces uncertainty (Keen and 
Meet orton, 1978, of #4). Bayesian techniques) orovide 
Porta | methedoloaies fOompe Tanabyzing, the imolications of a 
Metms1i0on maker's subjective judgment of Probabilities, 
uoOdatina these assesstents a3 additional) im fOrmatiomen 1S 
obtained. Pic agdtt tonal intormation can result in one of 
two events: either the uncertainty 18 comoletelyvy removed or 
mmeracdgditional intormnmation allows the decisicn maker to 
revise the initial (forior) assessment of the orohbabilities 
Mmeeem anc Scott Morton, 1978, no. 46). 

As stated earlier, the use of decisior analysis does 


Met reolace the decision mater hut adgs a new dimension to 
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Mpemrtecyision making caoabilities. It 15 also assumed the 
use of tnese manual techniaues could slow the decision 
Perea oroaress consiogcerably, esrecially {eS fear of 
mathematics exists. Fortunately, what nas neen develoved 
are Bee) S10N aids which areatly ease tne complexity 
mavolived. 

A decision aid 18 a human-systen interface desiaqned for 
Mmeemesoecitic Durerose of supoOortina and enhancing a decision 
meme 61S )6«6©6GeCISION Takina rcle (Keen and Scott Morton, 
Meeweeoe O35). It 1s a tool for use by the decision maker. 
Decision aids are mormally stcresn on commuters as the use of 
the comouter reduces al] needs forme ca rCUlations by the 
meson maker and helos to speed the decision process. 

The seneral availability of low cost, Bc cavacity, 
past [MerOormMation orocessina technoloacy has enabled man to 
Pem@emraas increase his intellectual cacacity. ites et fect 
Semmes Nas been that man can now deal more effectively with 
memmrex matters on hiS Own comolex terms a8 aaainst the more 
Beemolistic terms oO f ore=-information technoloay man 
Mememmers, 197e, op. 5). %The focus if not on the caomouter 
itself, but on the technoleay for dealing with the 
Motormation that is available and the scientific methods 
which are in tdevelorment and use. 

The use of decision aids nornally olaces increased 
Jemands upon human creativity and judament FacMmem= ec tam 
relievina the decision traker of the need to exercise them as 


ment be exoectec. Peirsaeiscmpecause the availability and use 





Seemeeerersior 3195S which serve 35 an extension of the decision 
maker's intellect provide new and interesting opnportunities 
memeusimec is tntelliect. Pecision aids helo the oaecision 
Mmmeere Oy i o2rescrihin2a 4a straianhtforward normative orocedure 
Meeteeien1 2199 3nd analvzina difficult decision oroolems 
Mepeeevima icoth uncertainty about Ene Ooutrcome “Of. future 
events and cernlexity about the complex value tradeoffs 
Mme in “he choice of a ceurse of action (Brinkers, 


mai c, DP » a. 
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@eammanc, control and communications (C3) can de defined 
Memes process which orovides the commander (or decision 
maker) with a means of receiving information, making 
decisions based aon this intormation, and then imolementing 
Bra monitorina the tecision 1n order to achieve his or ner 
mresion (o0ose, Sis Oe 5 C3 system includes the data 
Meeyerina (or inteliiaence) erocess, the decision making 
mrocessSr and the ooerating forces (or weaoons) reauired to 
Bepgeve the mission. 4 C35 system would look somewhat 1J1ke 


the figure shown nelo-. 


CONTROL COMMAND 


COMMAND |[_ 
COMMUNICATIONS,) centers [COMMUNICATIONS 








WEAPON 
DATA SYSTEMS 
GATHERING MANEUVER 
UNITS 





FORCE STATUS, DAMAGE ASSESSMENT 


PeEGURE. 6:: Goro Sem 
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ies fiaqure shows fthat data 1S aatnerea, possibly 
merouan the use of Salve narmima racers, chotoqraohy, or 
meets froOtT Within, and sent to a commanad center. Once 
within BeenGOQirsama “center, the data 1S analvzed and a dae- 
€1S1909 1S made reacarding the use of weaMPons or units. These 
decisions Tem inene transmitted to the respective units for 
Bareeementation. Reaquirements for data or information could 
Srepainate 3f the command center or it could he soontane- 
mmemeyeoenerated as in the case of a missile attack. 

linfortunately, Eye Tayo Gr 1 ty Gat ieee weal tof the 
Mesourcess spent 31m the imorovement of these systems are 
mmeeigieG 1N elther oroviding cetter data aatherina (such as 
better radars), Peluso eeCOmMmMuUMICationmsSs,. Of bigger and 
better weancns systems. These are all important aspects and 
Moc Oe iroroved, put this 1s also indicative of the lack 
of aopreciation for what must be done with the data when it 
Meme yvered to the command center. Little thoucht is given 
momemow the decision maker processes the the data.in seacnmnts 
meee ffect.ive cecisions and forecasts. 

fem amount of data available today to decision makers 
Remmemaooaclinag. Fealistically, if mnethods for cognitive 
information orocessina are not improved at the same rate as 
Mc omMUNICations can orovide its muchs 1f not all of our 
moonmisticated CGmaumications tHechmeo hocy a wi I) Bao “wader 
mili zed. what is neeged iS more research aimed at 
imorovina our iinfornation managements, decision making, and 


forecasting utilizina the anonlications available througn the 
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mermouter (4ndrioler, i920, o. eas 

Meersion rakina lies at the heart of the C3 process 
(A4rdriole, Lo, Oe le). The entire C5 system focuses on 
Srev~17dyna a reans to transmit al] requireaq data to the 
meGrsivon Taker and then a means to transmit the decision to 
the reauired oarties. Therefore, C3 svstems which do not 
Mmenmtain sore sort of decision making sureport should be 
challenzea because they are incomolete. 


Most aecisicn raxers will begin with a need for some 


meemOrOund infornation on the oroblem at Peand. Has tnere 
been eee oroablem lixe this oefore? HOw nave the people 
involved reacted rn Siem yal a ts Sit Wok icons © Are there any 


peagale available who have exnerience in this area? [hese 
are all exatroles of the tvoes of questions whichr wil! need 
to be answered. 

emt, te Jecisian maker will want all ot tie latest 
Meets Surroundinjza the oroblen. “tho 1S available to nove 
into the area? Howe ong «wi || ite =taovee Nno 38 there 
Mmireacy: “hat forces does the apposina sige nave available? 
Nhat Mom haloening now? These and many more are the 
questions surroundinoga the current situation. 

Now, the decision maker will want to know what the 
options are and what the adversarieS ontions are. In 
additions will be the need to know of any activities, either 
iN erocessS Or vDEeNdING, which mav nave an impact on the 
MememrsyOn. Vext is the assessment of the orobabilities of 


mee oOossinbnlie outcores” of these activities. ime criteria 


e7 


Which will be used in analyzing tne alternatives must be 
listed and relative itTcsortance weiahts for these determined. 
Summemiy the decision naxer syvyaluates the alternatives = and 
Memes a2 Course of acricn to be followed. 

How does the decision maker do alt lari), Siz esoecially 
Memeeerne iIi7cectrs of a wrona decision can be so costly? It 
mertainiy can not be tone alone. The decision maker must 
Pely on the aid of key staff members who have certain 
exoertises In snecific areas. But even with caomoetent 
Eeaffs, the task is often too areat. Comouterizeqd decision 
mmes tay be the only ee ne 

There are several reasons why comouterizedqd decision 


9812S woula ve useful [hie LC SeearGeci sions "mMa<inad “orocess 


mmmren et al. 1976, co. Aas Dig lmhese = Feasons are: 


Il. fat ions. Ee ttective tool for Tmerovina the 
Ses odue among staf f Siirecms who oce workeine 
Beate Of a marticular oroblem. It would be 


feroru) 1m eliminating misunderstandinas. 


feeime Orocess of uSing a decision aid gives” staff 
officers the onportunityv to include what they 
considers important in the analysis. Trent me y 
Gan see how these factors actually imoact on the 


Sout 1 OM. 


Se it 1s very ?asv to evaluate several courses of 
aetrrvaon. Variables which are Uncertain Cam) be 


chanaed to allow for the hest and worst cases in 


ae 





There are nutrerous examples of C3 problems whe 


ne analvsis. 


Peiticat Sreacmmcanm How be focused on. Si) tao 
nften, too ruch time 18 wasten In discussing or 
Seouina what MAY seem to he imnortant points, 
were tiem out to he rather tinsianificant ie a 


mae! solution. 


mee Tray Mon Fe In your favor ratner than your 
aoconent. Vost oenole do not react well when 
Hecer time constraints. PeOeile: stead tor szover 
React to. Unmimmoortant events ana make quick and 
hastv judaments. vVUecision aids enable ana force 
the meCcisION Maker to follow a systematic 
Roltine. Tnen the aid rerforns the necessary 
eeculations im 4 feal time manner. This aives 
additional time to either wait for additional 
[Mme Orcmation or actively seek POG int if the 


maa) ysis Shows a need. 


fe 


S 


momouterizeds decision aids would be useful and the following 


are some examoles: 


Waersiom 3916S Cam and are bheing used to monitor 
early warnina systems for any indications of 
moeemarag criSiS. ies Imoomeams CO NOt. allow 
Svemts to contro!) USs Hit wratners (Oo wess lb) y 


have usS Shane the events Cingr) ole, 1960, ie 


ea 








Nm 
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A recent examnle cancerned the analysis of 
jifferenr Sv secilat Tor DpoSotures that a commander 
PeemecoOnsiderina in tne tace of uncertainty ahout 
a develoningan crisis that couls have made it 
mecessary to evacuate U.S. Nationals from a 


if a aa, eountry Cina nS, 1978, <2) 6 Cle 


The normal ao wnaemMeyueolanmntag process that 
@omstantily takes olace 1S anotner examole 
memmlians, 1978, Ses em). [hemmmeeci sion a1 


femmes) YOU tc easily clay the "what if" aame. 


Mpoermina is another examole. By allowing staff 
officers the chance to use the aids based on 
Best crises, they becone familiar with current 
plans anc procedures ands if realistically used, 
Saeeone Usec! to Oceratina In a crisis environment 


Decision aidS may be used for establishing 
melicy. Qotions can be evaluated and tested 


more tO deciding on firm colicy. 


The desian of new C3 systems would be another 
examole. whenever new systems are aesianed, 
there 1S always a tradeoff of caoahbilities 
necause of Wiaolt fe went Fite a ac. These aids 


wouled helo getermine the safest cuts mn 
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teem CS DECISION AINS 


This chaoter describes several ee computer-based 
MIfTOFMmMations, decisionr and forecasting systems desianed to 
mepeeerne nunan user with the information so critical to 
efficient informaticn manaacement. 

There oresently exists much too little emohasis upon 
Ene content and esoecially the form of the im ormati1Tom ‘and 
Soca that Piowe tnreouen © 5 svysters (Andricole, 1°80, o. S% 
The questions of how the information should aopear to the 
user, how the infornation should de storea, retrieved, and 
mamyoulated, ana how the ever increasina amounts o f 
information can be comprehended by the user have not been 
moeaguateiy dealt with un to this point. The three 
computer=0ased infornation systems described below were 
develooed with these cquestions in mind. Each 15 aimed at 
helpina tne C3 system users deal with the enormous amount of 
merorration for which thev are responsible. Tne systems 
were develooed with the human user in mind and thus exploit 
mero supnliement existing human Iimtocmat tom processing 
caoabilities. 

Soatial Data Base Management = The use of normal data 
hbase manaszement Systems, from the onersnective of the 
Mmeesi ons! user, are difficult and inefficient Cimcir1.0 Fes 
mn 0, OO. 3). Recentlyr research has oroduced a prototyoe 


sratial Sata nanagenent System which enables users Eo 


Oe 





hierarchically store, retrieve, and maninulate data through 
meer use of soatial controls (Andriole, OO C ae. These 
Boatial controls allow the user access to data without the 
need or use of the conventional Keyboard. Instead, the 
soatial SemenolS cermit access to data using numerous 
meager yons! aids, such as colorr location (in an electronic 
momesoace orojected ona large screen display), touch, and 
Sound . Soecifically, the user may store and retrieve the 
Mae accordina to where he or she finds it most easy to 
mGCceSS. 

mae sOetial data manacement system was develooed iin 
accordance with the way humans normally store and retrieve 
maurormation. FOr SxaTroler in a normal office environment, 
meee store infcrmation in familiar places according to 
frequency of use, imoortance, shacer, $1iZ@s and so forth. 
The orototvoee soatial data base mananaement system uses these 
@eyocnologicsal eredisnositions by allowing the uSer to create 
mepemor her Own oSvcholooical worksnace and oroject it anto a 
larse disolay. This aisplay then 1s interactive and caoable 
Sree storing al] eyees= vot data to ime ogee numeric, 
Smotoaranohic, and even audio. [t 18 also eaSy to update. 
Pre user 31s then able to manioulate and retrieve data by 
MOVING Eanrouan the worxsorace with joysticks or towe 
sensitive disolav panels. Figure 7 shows the hierarchical 
data GG@ami zation orinciole. Figure @& showS a user 
BOsitioned in front of ae soatial data base management 


Meee. Figure 9 shows the jovstick and touch panel mounted 
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FIGURE 7: HIERARCHICAL DATA STRUCTURE 
{(Andriole, 1980, P. 5) 
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FIGURE 9: DATA ACCESS CONTROLS 
(Andriole, 1980, P. 6) 
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» bdid pad? 


momeeoeorr Of 4 USer’S Chair. Data 18 accessed by touchina 
BesPecific Soint in the worksoace or the touch-sensitive pad 
Seereorremoy ""™OVING the jovstick forward or oackwara (to ascend 
MM—m~m@iescemd ints tke hierarchically organized data base). 

meeerve C4 Information Selection - In resnonse to the 
Perm increasing information requirements olaced uDON 
Mieermmiacence officers responsible for analyzing and routing 
memo m@atyonr the Edantive Information Selector (AIS) was 
memweroney (“adni et al., Loo oie lis ee Die Vaart 1S 
develonmrment orocess, ‘ati WaS determined that "new Gs 
mememmvates are requiren to control information flow so as to 
Meeeematcc). svster cacability with human characteristics in 
mrem@an=comeuter interaction”. 

ieee lS orocess is as follows. The system user is 
Bevorated during a training ohase to determine how and why 
some messaaqes (eveces, JOf information) are selected for 
Meecner analysis and some are rejected. This 18S performed 
for eacn ieee iedal= user such that he or she has 2a 
Bersonalized inforration management SorulnuyseroutInNa, and 
Queuina systen. 

What hapoens, in effects 1$ the computer monitors the 
user and then jinternalizes how the user selects and rejects 
merormation. Once the training orogram 1S accomolished, the 
momouter ettulates the user and automatically selects, 
routes, and queves the information. Thus tne AIS may be 
viewed as an information management assistant canacle of 


reljevina its syoerior from some. of the burdens) of 


oy 





information overload. 
ihe mune t1Oons amic elemants Gt an S1S are oresented 


Memyomicalily in Fioure 10. 
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FIGURE 10: ADAPTIVE INFORMATION SELECTION 
SYSTEM CONCEPT siananiole = 1980, 
P. 8) 


The AIS has been develoned for use where C3 information 
is already imeean electronic Torm. The AIS is designed to 
Supcort areas which are already comolicated with CoCr anuice h 
incoming information for the human user to handle. The AIS 
is merely a comouter oroaram which orovides helo to tne eS 


systen user. 


Ultra-Ranid Readina ~- Even if systems such as soatial 
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Pema ase mananenent and AIS can auickliyvy orovide access to 
Meeeteamyntornation, the user nust still reag ann comorehend 
tne mere ermatyion nefore further action can he contemolated. 
Suoseauentiy, research has neen performed to irmorove the 
Smee ae with which users can read and comorehend (Andriole, 
meee oo. Fn-il). 

A technique known as raoida serial visual OResemt at lem 
(RSVP) has peen aeveloned and comouterized for use in C3 
Svstenms. ieee ly Sx t 1S oresented one word at a time on a 


memmeeome fray ture (CYT) in Rae SucceSSsION as Fiaure | 


sugaests. 





PYrGuUeeEe hi: ULTRA~RAPID READING 
(Andriole, 1980, P. 10) 
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Research has Meate@seed that sindie RSVP sentences can 
Memes? Aan accurately recalled when shown at a rate as hiah 
Semmens Or%S Cer second, which i$ twice as fast as people 
meee y read (Kotter et al., 1979, o. Io. 

Readinq sneeyd and comorehension is a function of the 
Mmmemert tares for the eves to transmit to the brain and the 
Memes ability to orocess. pue TUltra=csao01a reader was 
develoned to shorten the time hetween the eyes and the 
mein. ft does this bv reaqucina eve moverent to a bare 
aires «6d Ne 68yesS dO NOt meed to move back and forth or uo 
and down, as 1s normally doner since the text aopears on the 
moaeume the Sane jocation. Guaicl <! iet at Oiny Substituting some 
mremures Or Ssy™bcls for words haS INcreasen comorehension. 
However, aS with ATS, the text must ne in electronic form to 
Semeread ultratmranidliy. 

The preceding three researcn desians were dveected 
Seward ImmOrovina the tinfornation flow to the deciston maker. 
The next three examoles are computerized aecision aids which 
meee aesiGgned to te used crimarily during crisis situations. 
Each permits the user to select and retrieve history on past 
Mersis situations. The best way to describe each 18 by 
providing a samole of the tyoes of aueries which are 
allowed. tee first, AS Gesecioed In poure les, enables a 
Meer to retrieve and orescrihe crisis actions (or resnonses) 
and objectives. Ihe second ai4, as shown in Fiaqure 13, 
enables a user to focus on the manaqaement croblems which 


mecurred Guennc Sast ¢erises. The third aid enatrles a user 
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Moexamine 507 crises involving the Jeo. Det ween 1946 ang 
myo, as Fisure t4 shows. 

These ai131S were develooed Limo Soierec tk response to 
requests ron crisis managers for sone type of on=line 
analytical assistance (Andriale, “Way-June 1979, Oi I) ce 
piers managers are now able to assemble, sort, and analyze 
crisis manaaenent data ona real time basis. Freliminary 
Bepformerce tests of these aids sucaest they miaght increase 
the ranae of oction aeneration and Svoluoguiven durrma crists 


by 100% and reduce oast search case RipmemGy. 93.0.4 


THIS SECTION IS DESIGNED TO ASSIST DEPARTMENT OF DEFENSE PERSONNEL IN 
EVALUATING PROPOSED COURSES OF ACTION AND SETS OF U.S. OBJECTIVES BASED 
ON DATA FROM 101 CRISES INVOLVING THE UNITED STATES BETWEEN 1956-1976. 


THE USER IS GIVEN THREE LEVELS OF ANALYTIC ASSISTANCE: 


1. THE CAPACITY TO SEARCH FOR HISTORICAL CASES WITH 
SETS OF USER-SPECIFIED U.S. ACTIONS OR OBJECTIVES. 


2. THE CAPACITY TO IDENTIFY, ACROSS ALL CASES. 
THOSE ACTIONS THAT HAVE HISTORICALLY BEEN 
MOST STRONGLY ASSOCIATED WITH EACH OBJECTIVE 
SELECTED BY THE PROGRAM-USER; 


3. THE CAPACITY TO IDENTIFY, ACROSS ALL CASES, 
THOSE ACTIONS THAT HAVE HISTORICALLY BEEN MOST 
COMMONLY ASSOCIATED WITH SETS OF U.S. OBJECTIVES 
SELECTED BY THE PROGRAM-USER. 


WOULD YOU UKE TO SEE A SUMMARY OF ANOTHER SYSTEM SECTION? 
PRESS ‘Y’ OR ‘N’ AND ‘RETURN’. 

iy 

ENTER ‘I’, ‘ll’, OR ‘HI’. 

il 


FIGURE 12: CRISIS MANAGEMENT DECISION AID 
(Anadrtole, 1979, P. 16) 
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THIS SECTION PERMITS A DETAILED EXAMINATION OF MANAGEMENT PROBLEMS 
ENCOUNTERED IN 41 SELECTED CASES (1956-1976). 


MAJOR PROBLEM CATEGORIES ARE: 


SYSTEM-RELATED DELAYS IN DECISION-MAKING 
SYSTEM/PROCEDURAL CONSTRAINTS ON ACTIONS 

LEGAL ISSUES INVOLVED 

RESOURCES INADEQUATE FOR DECISION-MAKING/ACTION 
INTELLIGENCE FAILURES AT DECISION-MAKING LEVEL 
EMOTIONAL/IDEQLOGICAL ISSUES INVOLVED IN DECISIONS 
INTERPERSONAL FACTORS IN DECISION-MAKING 
PROLONGED CRISIS PROBLEMS 

PROBLEMS IN SELECTING ACTION PERSONNEL 

10. CONSTRAINTS ON OPERATIONS 

11. PHYSIOLOGICAL PROBLEMS FOR OPERATING FORCES 

12. INFORMATION FAILURES BY OPERATING FORCES 

13. FAILURESIN TAKING APPROPRIATE/TIMELY ACTION 

14. FORSTAT PROBLEMS 

15. PROBLEMS IN THE OPERATING ENVIRONMENT 

16. GENERAL PROBLEMS IN CRISIS PLANNING 

17. GENERAL PROBLEMS IN CRISIS HANDLING 

18. GENERAL PROBLEMS IN CRISIS TIMING 


WOULD YOU LUKE TO SEE A SUMMARY OF ANOTHER SYSTEM SECTION? 
PRESS ‘Y’ OR ‘N’ AND ‘RETURN’. 


ee See 


Y 
ENTER ‘I’, ‘Il’, OR “HI. 
Ht 


FIGURE 13: CRISIS MANAGEMENT PROBLEM ANALYZER 
woancdmeno le, 1979. PP. 17) 
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THIS SECTION PROVIDES ACCESS TO INFORMATION ON 307 CRISES (1346-1976). 
THE DATA MAY BE SEARCHED FOR CASES MATCHING THE USER’S SPECIFICATIONS, 
OR A FULL DESCRIPTION MAY BE PRINTED FOR ANY SELECTED CASE. 


CATEGORIES OF INFORMATION CODED FOR EACH CRISIS ARE: 


@ YEAR AND BRIEF DESCRIPTION 
@ LOCATION OF CRISIS 

@ ‘NATURE OR PRE-CRISIS ACTIVITY 

@ DURATION OF PRE-CRISIS PERIOD 

@ SCOPE OF CRISIS (DOMESTIC OR INTERNATIONAL) 

@ NATURE OF CRISIS (MILITARY, POLITICAL, BOTH) 

@ CRISIS DURATION 

@ TIMING OF CRISIS RESOLUTION 

@ CRISIS OUTCOME 

@ ANTICIPATION OF CRISIS 

@® DEGREE OF THREAT TOUSS. INTERESTS 

@ TIMING OF THREAT DEVELOPMENT 

@ TIME AVAILABLE FOR DECISION 

@ SIZES OF PARTICIPANTS/ DEGREE OF U.S. INTERESTS 
@ U.S. RESPONSE/PARTICIPATION 

@ U.S. OBJECTIVES 

@ NUCLEAR/NON-NUCLEAR IMPLICATIONS 


WOULD YOU LIKE TO SEE A SUMMARY OF ANOTHER SYSTEM SECTION? 


PRESS ‘Y’ OR 'N’ AND ‘RETURN’. 
N 


FIGURE 14: CRiSts OR SceCREETOR 
(Andriole, 1979, P. 17) 
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Sreina crisis situations, decision makers strive to 
react Swiftliv, decite wisely, and conmunicate accurately. 
mes reaquires close coordination with their staffs and the 
Bomrty to overcome certain onstacles such as oressures from 
mmemmGonstraintsre anhbiauity of goalS-r and the monorolization 
meeetme with information collection. Some of the problems 
of information collection nave been solved or at least aided 
Steme, use of one or more of the oreviously discussed aids. 
fihat the aecision aker now neerts are effective decision 
strateqies that Veeoose Figor ands orovide a4 HOI es lly 
moeructural framework to assist them ino ene —sorocess. “of 
Smoosing an ootita) decision alternative in the face of 
Volurinous ang often inconclusive evidence. 

Mereations and Intellicence (OPINT) 18 a decision too} 
Bhat orovides just Suen a0 tramewor < for odeliteration, 
Mmeeeomima, and analysis (Amey et aler 1979, o. 43). OPINT 
memam On-line, interactive, real-time model whicn aids 
Meersion makers by orescrihing a straightforwara nornative 
Semeeaure for orgqanizin2a anid analyzing difficult deciston 
orotnlems. Ptecemorobilens tay tnvolve booth uncertainty about 
the outcone of future events and perolexity about the 
complex value trade-offs involved in the choice of a course 
Beaction. 

Seni) 16 a decisian-analvtic basea, comouter assisted 
decision aida. Les Be any objective is to provide decision 
memxers a oroceaural frameworks, or decision template, that 


insures their ultimate decision choice 18 a coherent cone. A 


Gu 





momerent cnoice icmmame whe? 1S comsistent with their own 
meee striictures and heliefs about the relative likelihoods 
Peertire events that wil! imoact the decision outcome (Amey 
ees, LIS, Of Se 

The fundanental oroduct of OPINT is a comouter-stored 
monceotusal reoresentation, or decision model, on the 
decision oroblen at hand. fhereas decision analysis 
Brovicges the thenretical background and procedural guidance, 
the Dl it dee1sion mode | erovioEes the SOGC I hc 
methodological Ba0 ) for orocessina information and 
evaluatina the various decisicn alternatives omen to the 
mecision raker. Me inedenth exolanation of how OFINT aids 
memeeye gecision orncess 18 discussed in Chacter Four. 

meerruation (EVAL), another onwline, interactive, 
realtime model, incoroorates another decision strategy 
mmpememorovides the cecision maker with a normative procedure 
meme malyzing daifficult decision problems. Tt 18 eSsoecially 
desianed to evaluate and comnare possible options using a 
meme ye-attribute utility analysis techmraue. EVAL 4s best 
amnlied to orocurement decisions wherein Systems are 
compared, but is anoronpriate for a diverse set of decision 
analytic oroblems (Decisions and Desians, omnerene Gov OO. 
>). 

The use of EVAL is broken into two Dhases. First, the 
decision maker creates the structures of a hierarchical 
Memeepeattribute evaluation model using the EVAL Structure 


Brogram. Then, this model is evaluated and processed using 
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mime EVAL oroaram. 

Memeeorder to utilize the EVAL Structure orooram, tne 
mimoe'en “Sst be decomnosed irto 934 hierarchical Seruceure 
which merlects the MOQECS| Vaterrelatlensm1o. of 411 the 
maGeors involved. This structure is then entered and the 
mode | ibommecreated, NOu Under the Girection of the EVAL 
progravr, the criterion used for evaluatina the alternatives 
are entered. Onee entered, their resoective weiahts of 
imrortance are entered. Next, the relative score which eacn 
Meeemmative achieved for each criteria is entered. For 
examole, consider tnree radio sets which are being evaluated 
for selection hbefcre onorocurement CAL, Ae, and AS). The 
eriterion of evaluation are cost, range, portability, and 
Seriadility. Each of these criteria are weighted 30%, 20%, 
10%, and 40% resnectively. “ext, the three radio sets are 
_—=Emnietiecd accaraqina to each criteria wusina an arnitrary 
Mmimrity scale. Cost would show scores of 59 for set Al, 4) 
more set Ac, and 8&0 for sat AS. This means A434 is considered 
Fo have the hest cost. This orocess 18 reneated for each 
meiteria. hs is also normal for the criteria to have 
Subtcriteria which would also require the same procedure. 

Next, EV4L aives the evaluation scores for each of the 
alternative systems resultiag from the acgaregations 
ePrescribed in the model structure. Now tne decision maker 
Can examine the effect of the criterion weights throuach a 
sensitivity analysis. Peas will quickly Show him which 


criteria are the critical ones and take action to inSure the 
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data Bur rOUuUNnnIiNa witch teria 16 as accurate as oossible. 


Pempemeaoes not do anvthina the decision mater himself Gould 


Not have Pome 11 T Se lots (tMGtnooe Sega. however, 15 41] low 
him not tO oe oressed Oy time Or to worry apout 
comoutations| procedures. memmGam Olay tne @awhat i1f' aame 


memerree | Confident about the decision he must make. 

Another decision aid has been developea to assist 
meoeos 17 Veciding how to deal WieGne 1 MO Oman t issues or 
orodlens. PeSecuUrDOose = 97S tne I1moroverent of decision making 
Memeo yINU0US Marticicant Wateraciton wer =the <comouter 
Mmemcmee the decision mnakina crocess (Leal et aler 1978, oo. 
Pee “embers of the aroun are allowed to inout their 
respective estimates of the occurrence of snecific events 
GmermetPeir values (weishts) of the imnortance of snmecific 
Meeisvon outcomes. Tne decision aid then comoares the inout 
and informs tne arouo menroders of any disaaqreements hetween 
them. Bure most imoortantliyr it focuses the discussion only 
on the disaqreenrents which are inoortant to the stated aoal. 
mem ytterences which are not ivroortant are shown to be just 
meee Finally, it suaqests to the grouo crecisely how their 
disaqreements miaht he resolved. This enables grouos to 
move much more quickly toward decisions. AS a by=Droduct, 
profiles are develoved of the particivants which show who 
has exoertise in snecial areas as well as who 18 a 
Mpei=avoider or risk-taker (Leal et ale, 1978. mmo. 173 to 


mor). io tunertvonnof tne aid is shown in Figure t5. 
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PROCEDURAL 
INSTRUCTIONS ANDO 
DATA ON GROUP 


CONFUCTS 
DIRECT COMMUNICATION TO 


INDIVIOUAL PARTICIPANTS 
VIA INDICATOR UGHTS 





pescrise Weer ENTERS UTIUTIES, PROBABILITIES, 

ALTERNATIVE “Gece AND VOTES 

DECISIONS/ 
EVENTS 


j ENTERS DECISIONS/ 
EVENTS ANDO MANIPULATES 
DATA AND DISPLAYS 


PEGURE 15: PRBS GROUP “BECTSLON ALD PROCESS 
CA nee. hoy Oe. ky) 


The decision aroun is composed of the particioants Cde- 
mysion rakers), an internrediator, and 3 director. 
Dime intermediator's Or inory fuce t 10m Ss tc fae i= 
itate cormunication between the aroun) and the computer. 
This is done kv translatina snoken requests foto Feomout = 
Mem yAputs. These innmuts could be lists of events, alterna- 
Pave actions, reauests for displays,r oir modifications 
to any orevious inputs. The director helos with the outout 
by oresentina and explainina it to the oparticioants.e 
Meee director also focuses the group's activities and insures 
mime inouts are anpcropriate. As a result, virtually no 


marticicant trainina is needea to operate the system, and a 
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meoure Can beqin work con its decision proolem almost 
immegqiately after it is convened. 

Baother computerized ey tre Jharly Warning and 
Monitorina Svster (EH#AMS), has Deen oavelored to aid the 
meeear rons and Warnina (ISH) community. EV:AMS orovides 
them with an interactive conputer based system of oolitical, 
militaryvs and domestic leave juemsaltoreda) | Yamon’ cor) ma of 
Beeermational and intra national ae tas (Daly, Lo 735 os 
a). The system nas the cacability tc store quantitative 
mamememationa!l oolitical dat3 from t966 for most countries of 


tne worled S\rvel! emsia: therefore nme usec to do retrosvoective 


analysis as wel | as CuGcremts aay Taw. The data 1s 
aqaqgregated ny rontkh, Santer, and year. EwAMS has the 
Samability to feiac K single countries, combinations of 


moumtry~Oairs, reqions or the entire world with these 
meat ytative indicators, or  soecific reaions May be 
soecified (Daly, lo 27 Dee OJ. S@ ty presents, there 1s some 
doubt as to whetner EWAMS will ever be fieldea because its 
mmeration is extrenely slow. But the concept 18 a good one 
and further research may prove vceneficial. 

lime EHWA4MS is c¢conorised of the toelvewing “comoonents 


memgriole, 1930, on. 28-34): 


ee General) Scans - 4 aeneral scan 18 an aagregation 
of countries Dy sone Eriteriay Betas 3 scan 
defined bv yeograohic region would aqgreaate 


eoumenries by aQrouos such as Latin America Outs 


Middle Fastern. A qeneral scan allows) an 
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analyst Wow book at several] countries taken 
together rather tran lookina separately at many 
@aumcery-rmaAirs. aso mina cat 1Ons tore the Grouc 
as 3 whol> suagest UmUStia |! lyr Miran sctivatys 
memioyons, OF uncertaintvs, the analvst can drop to 
mie CGOoUntry=hy-ccuntry level or even track the 
meicemu activity of a simalie country to determine 
tne source of the disturbance or the regional 
level. General scans thus make the svstem much 
more efficient and less time consumina for the 


analyst. 


PGantitative, Politicale Military, Economic, and 
Pemestic Indicators - Quantitative indicators 
Boece risSiS warning include foreianse domestic, 
omae international SQ wee 1 C ale; nme vtary + and 
Seeman 1c faetor Ss. The indicators are both 


dynamic Caventis=s=aaSeo ) anc St ot we Cattriouie 


Based). 
moh t 4 “Tet hog. Forecast ime  Gapad1!17t y - The 
unified multiwmethod fomecas ( WhiceeecaPpao) | tty 


requires the system to generate different kinds 
Of forecasts or warninas via different methods 
Vonoeeedytferent events and conditions.. Scan 
oo0tions have been desianed with reference to 
eeilectae,. Gdoalse and mnetrods of forecasting. 


Experirentation is still beina overformed in this 


oe 





af ©) « 


Peay tyonal research 1S clurrently being nerformerc i> 
Meerom 3M 319 which will) focus On Soviet Ef1S1s behavior 
memee world ar lee This aid will include the following 


tasks: 


Ps Develoomnment On an lAvier tor yuo tascoyv)et CfilsS 146 


management behavior, 1946-1979, 


Ce Collecticn ot aqata on Bre we taraeverthst ics cof 
these crises iO Show the nature of Soviet 


military crises. 


fee TOre intensive analysis and coding of key 
Soviet crises fo identify the Cari SS 
environments that mav affect the occurrence of 
crisiS management orobdlems encountered by the 
pevyet Unions and variables describina crisis 


oojyectives, Se tionswrana nestmilts tor the Soviet 


Union. 

1 prot ystical analyses Cte enerecnorocterist ics ‘oF 
Soviet Eris is manaaqement Soerat ions, the 
environment jm whieh they have taken olace, and 


PaNew Crisis management oroolems encountered by 


mee ooviet Union. 


Pmeeeomoarison of these statistical analyses with 


earlier analvses 0 f the same factors for the 


= 





eee! 


pel] ot these decision ais have been aaovertised as 


Beemer valuahbnlie to ‘dtecision nakina. None nave been 


4 


experimentally evaluated with knewledaeable sunjects ina 
Mecision raking environment. A baseline of such evaluations 
Bemeemoorcant to establish in order to provide a useful too) 
and better yet, Boul oroy1ac Ore Muic a needeaqd technology 
transfer and real sorld, real time feedrack on the value 
mmmmrry) of the tecnranoloay. 

Se! was selected aS an aceropriate candidate for 
experiment3tion for a numner of reasons. EWES ky vee 1s 
readily availaole as a odackage at tne Maval Postaraduate 
Seton Second, all cof the equipment required for its 
@eeratyion already exists at thie oC HOG sme wna lly, -aue to 
both tine and monetary constraints, the procurement of other 


eras was unfeasible. 
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erste sk DFSCRIPTIGnm OF OPINT 


Mer 1S a Cecision~aralytic based, comoutersassisted 
memmston Sstrateaqay (Amey et al.r-1979, oo. A) It was 
megyaned and Suilt hy Decisinns and Nesians Inc., of McLean, 
Virainiar, under contract to the Nefense Advanced Research 
Projects Acency (On P a). rat was written ime Chie APL 
orogcramnmninga lanauaae fore the li St0U conouter. In 
additions ewtil) has recently been rewritten im FORTRAN UP 
memeetuise On FDP 11 series comouters. 

fmeeeeceneral ournose of OPLIUT is to aid decision makers 
ByeProviding then a4 canabhility to Cons eruct, store, 
Mameve, exerciser, and refine decision~analytic models of 
complex decision croblems they face. The OPIMT decision 
model, which iS interactive, reale=timer, and on-line, 18 an 
Mmmeqmamyz1ng franework for information ofrocessing. Decision 
merry sis 15S a methodological tool with which the decision 
maker defines and exercises the OQOPINT model to evaluate 
Becision alternatives oertaining to the problem (Amey et 
mame 19/9 OO. See los. 

The OPINT system is cdesianed to be used interactively by 
decision makers who are relatively unsophisticated with 
Mmesoect to computer technology. Accordinaly, the desian 
Beristies two humanw-factors objectives: OPINT 18s a menu- 
driven system and is desianed to be foraivina of oro- 


cedural errors dy the user. 
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Bae decision model createc bv the user mav be aiven a 
mamawe label» although each is constructed usine the same 
aeneric Zormat. remrmoge |! 1orhat 1S Shown Gracnically in 
Bagure !6. 

mae tormat Goms ists on the following elements 
which, when completely specified, uniquely define an OPINT 


meci1sion node!: 


meeene® Jecision *- A ghort label definina the 
@ee1sion problem. This ltabe! is also applied to 
the decision model and 1$ used to store and 
retrieve tne nodel. 

eemeeDecision Mmeveraat tyes = Ss listo ~—deC1S)on 
eecermatives availanle to the decision maker. 
Eacn alternative is aporooriately labeled. 

6 mammeertain Future Event = A key uncertain 
event, a that will influence the eventual 
plume one of the decision. fhe wURecert ain even t 
is attached to each of the decision alternatives. 

G4. Event Qutcomes = A list of the discrete event 
Outconesre, each annorooriately labeled, Cert 
together define the universe of oossibilities 
reaardina the occurrence of the future event. 

Serevent Provabilities = 4 vector of orobahilities 
G@mabeare associated with the event outcomes, 
wach reoresent the orovability that an event 
will occur. A probability 18s 4a measure. of 


uncertainty. it 1-424 numoer between ) and ll, 
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inclusive, that reoresents the extent to which 

an individual helieves a future event will occur. 
However, WieeeaesS es SOeCG )TNGatClOm, Orobdah1)ities 

are exoresseqd aS a oercentaae of certainty, 

Pemcie, as 40% vice 0.4. 

6. Decision Qutcomes - The elements discussed thus 
feo define voossible decision outcomes. Each 
decision outcome is a vairad combination of one 


peeision alternative with one event outcome. 


The remaining three elements of the model format are 
used to soecify the relative conseauences associated with 
mre aec1S10nN outcomes. The consequence of an outcome is 
exoressedqd in terrs of the relative regret that would be 
exoerirenced bye the “cecision maker should the outcome 


mera iy occur. These three elemerts are: 


meme 1s1io 7 Cutceme Criteria = 4 list of criteria, 
maem aoocrorriately labeled, by which the 
decision maker would judaqe the relative reqret 
Besoctated with the decision outcome. 

Peeeeecriteria Neights = 4 Vector Of welants 
meciecrated with the criteriar which represent 
membre tative contribution “of the criterion. 
Criteria weiants are expressed numerically as 4 
oercentaae of the wholes, e.Ger aS O60%. 

3. Rearet - Regret is a measure of the consequence 


Ot a meas ion Outcome. The tote reaqret 


alc 





Bsesvones to 3 decision outcome 1s a welqhted 
linear Gamer at 1on of 18" Vay aia cer ltermia 
Pearets. oramoac h Eriteriom aime for eacn 


Mmecersion cutcome, the user must Soecify a value 


Siereqret. A reqret 18 a number between 0 and 
-l, inclusive, that reoresents tne relative 
meaeree of dissatisfaction pate bes dec 1s 10n 
maker associates with a Sarr ceu lar deeys ion 
outcome. 12m reoresents no rearet; =) 
renoresents maximum reaqret. However, im this 


Specification rearets are always exoressed as a 
pmercentaae of the maxirum; a. Ons Ss 6 U1 Ce 


macs 5 6 Peifer wuom: 1aqure 17. 


memomconoletes the nodel format. ine sere iis lon moce | 


eonoletely and uniquely specified when the elements 
described above are defined by the user. The input 


meee, fications AeSse pie inc the mode | can be processed to 


Sroduce the followings results? 


Ll. 


Compinead Value Rearet Matrix = 4 sinqle matrix 
fear adisolays the total or combinedaq rearet 
mescoeyated with each of the decision outcomes. 
For each outcome, the combined fegret is 
ebtained by weightina and aagding the component 
Mearets contributed by each criteria. A combined 
value rearet matrix is snown in Fiqure 18. 


Exnected Value “atrix - AN sinale matrix that 


ou 








CRITERION 2 -C WEIGHT W 
G 


q 





| EVENT “GUTCOMES 


| =, | z, | 


Gear TeRLON: C WEIGHT W 


PE Vialimo ou. COMES 


SerrERION: C WEIGHT W 


aff = Ta 


Fe . 


| at ein 


WY) 
fy 
O 
ei 
Oo 
on 
O 
a 
O 
KH 
” 
KH 
O 
fy 
Q 





MrGURE 7: REGRET MATRICE 
(Amey et al., 
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COMBINED VALUE 


E, (LABEL) E> (LABEL) Eg (LABEL) 
D, (LABEL) 0 -17 -31 
D, (LABEL) = -9 -9 -17 
Dz (LABEL) —-24 -22 -22 
D, (LABEL) 30 -26 -26 
FIGURE 18: A COMBINED VALUE REGRET MATRIX 


(Amey et al., 1979, 
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disoltavs the merartea expected rearet, 


associated with each Mocs Vom outcome, The 


expecten value matrix takes Melieve: account the 


relative Mice iaOoGgs of the evemt. outcomes. 


emeexmected Value Vector - A vector that aisolays 

the weiahted exnected rearet, associated with 
each of the decision alternatives. 

The exoecteo value matrix and the exoectedqa value 
eector {total ) Tecmo vsOayecd “toeaqgetmers aS ShOwnm in Figure 
i. 

itememmaere two sensitivity analyses that are usefyl to 
the user. Both are based on the exoected value vector and 
are described as follows: 


1. 


Threshold fiat bf 1 x > Am arte x Ci ote seso | Svs “ene 


eleven expected value vectors aenerated by 
eyemer one of the following onerations, at 
the user's oction: 
ie Varying the crobability of a aesiqnated 
event outcome, from 0 to 1004, in steos of 
ue The other event outcomes, maintain 
their pronortional relationshins with each 
Sune Tt « 
b. Varying the weight of a designated criterion 


from 0 to 109%, in steos of 10. Tne weights 
9 f the other Crisveriea mete So le their 


oronortional relationshio with each other in 
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Eee ep Valu 


Ey (LABEL) Eo (LABEL) Ez (LABEL) TOTAL 


D, (LABEL) 0 0 -4 4 
D> (LABEL) a: 0 -2 -7 
Dz (LABEL) = - 12 -1 3 -16 
Dy (LABEL) = -15 -1 4 ~20 


EeGuRE 19: AN EXPECTED VALUE MATRIX AND 
AN EXPECTED VALUE VECTOR (Amey 
Clea yy oo eee) 
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the sare manner 35S do the Promo m@illttaes 


fesertoeanina. the orececimGg oaracrann. 


In noth of the above cases, the least rearet disolaved 
moet me excected value  oayvof f vector 1S identified by 
Smeoate risk. ‘lormaliv, the decision alternative that leads 
momene.) 2eC1S10N outcome having the least rearet will change 
mseeume Jesianateao event probability or criterion weight 
iS aiIncremented ae Ueeet oo oe The cscoints of change 
MmemePetrerre) to as threshold points and are noted on the man 


mreix. mectire -<)) ShowWwS an examole threshold matrix. 


EXPECTED VALUE WHEN 
PROBABILITY OF E; (LABEL) IS: 


0 10 20 30 40 50 60 70 380 90 100 
eee eee) § -57 -52 -46 -40 -34 -29 -25*-17*-11* -6* 0 


Seeeeee)  — 8/ 42 -33 -34 -50 -25*-21 -17 -13 —-9 
Dz (LABEL) -27 -27 -27*-26*-26*-26*-25 -25 -24 -24 -24 


Dy (LABEL) -26*-26*-27*-27 -27 -28 -28 -28 -29 -29 -30 


FIGURE 20: A THRESHOLD MATRIX (Amey et al., 
Og oe eae) 
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santa l | y 


chanae event orokravdilities - The user 


mMmaV 3enNerate 4g test exnected value Vectar tient 


aS hasen 


On SGulaacelrrarlwly assiemec Vector of 


event Srpopaotlitires . The user may soecify 


Sever3l aire ce nt crobability vectors and note 


the resultant expected Yo lace Veetors. An 


Beamole of 


Sema sco lay)s oShowp. 1m igure cl. 


KOREAN INTENT 


NO ACTION HARASS SHOOT DOWN 


mere eNT LIKELIHOOD: 60 18. 0 
SeveaR REVISED VALUES: 30 60 10 


NEW VALUES: 


30 60 20 


fier nieok VALUES ARE CORRECT TYPE GO: (GO) 


EXPECTED VALUE/REGRET 


Dee NOT FLY =38 
Merri PrIED ROUTE =2.6 
PaeiaeD ESCORT ie 
HIGH PERFORMANCE A/C =—® 
NORMAL MISSION 29 


BrEGuRE 21; MANUALLY CHANGED EVENT PROBABILITIES 


(Amey ose teva. 7. 19797 -P. le} 


Ge TNT 1$ aesigned touoertorm {hes bas 1c functions 


@escr 


ls 


bed below: 


l1prery o f OP INT models ~ Store 


wienmtoain a 
various gec is ion models, filea py their 
associated labels. 


os) 





a. 


Load an existing OPINE model = Disolay the 
labels of those models stored in the model 
IoraeVY, and oermit the user to retrieve anv 
fresired model. The loaded model 18 referred to 
as the current model. 
Mapsoillay the results of the current model - 
Permit the user to examine the structure and 
Bemeremt of the current model by disolavina: 

Ae event orokabilities, 

pe eriteria weights, 

Ce rearets, 

Se combined value matrix, 

Crs exoected value matrixes, and 

mm exoected value vector. 
mewise the current model - Permit the user to 
make chanoes to the Structure and content of the 
euieremt nodel. The user may revise: 

ae event orobacilities, 

is erpiteria ama criteria weigqnts, 

eee decision alternatives, 

ele regrets, and 

e. combined value regret matrix. 
Save the current model - Permit the user to add 
the current model to the model library. 
Perform sensitivity analvses - Fermit the user 
mouerest the sensitivity ef the current model by 


determining thresholas or manually Ghana ma 
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probabilities. 

Mpeoste a mew NPINT mogel = Pernit the user to 
create a new mnodel, wT en then becomes the 
Current mouel. Tne user creates a model by 
scecifyina the elements required for the model 
as exolained earlier. The user may also use a 
reference aamble to validate the assianed values 


of regret. 
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Venice sim ov GN SOF THE EXPERIMENT 


Poe conduct of experiments using command and control 
mecr1si0m aicdss such as OPINT, present special propnlems (Cain 
maeeron, 19/8 op. Ba Hus tome ily yect he eval Uat 1om. of 
command and contro! systems has heen focused on hardware 
measurements. little attention was maid to software, 
memecta!ly decision aids. Ss a result, Se ES defined 
Silamtrtative Treasurenents exist for measurina the quality of 
Memmereemst1Om alds (Sinaiko, 1977, 20. 576). Seconda, there 
are no “correct" answers toa decision oroblem (Daniels, 
oe meee ather, an aid such as OPINT 16 a cognitive 
tool =O assist Emme “decision maker wm revealing and 
formulizina the elements of the decision problem asthe 
decision naker perceives them. Lastly, during the 
develococment of these decision aids, there has been 
relatively little syneraism between developers and potential 
meres (Cain and Poh, 1978, o. oh ae Users are often not 
familiar with the new technoloay and are unsure exactly what 
to expect in the way of performance. Nevelooers are unsure 
Meret ly what the ocotential users want. This occurs hecause 
they aren't sure who the users will be and causes a very 
qaenera] aid to he developed - one which will try to suit 
everyone's needs. 

Any exceriment which involves the use of decision aids 


must be desianed carefully. Tne experiment should be 
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meenemead) Im OfcCer ty assess both the capabilitv of the 
Meetmorocy and its overall utility as it relates he total 
Summer oe rtformance (Daniels, 1977, o. leah) & There are four 
Mevels Of exceritents varvina in fornralisn from very loose 
mrree Olay with only subjective judgqnent outout to "highly 
memvetrured with comolete snecification of conduct of trials 
And outout @onsistinc of carefully measuregq systen 
mmmmnomces (farr et aler 1978, 2. Ce). These levels’ fron 


m——reesGtmictured to most Structured are: 


1. Validation exnveriments. These exoeriments would 
e@onsist o f the deouaainga of hardware or 
software. The feasibility of the system to work 
Mee@ordina to fixed scecifications is determined. 
Amn example would ne the dehuaaqina of the OPINT 
software Seiorm uto ~olsacing vt <om Lines “tor 


@aerational testing or use. 


2. Demonstration experiments. These experiments 
Poula be somewhat nore structured thar the 
valication exneriment, in that a scenario would 
be followed. The tern demonstration refers to 
@pemetyoe of output from the exoceriment. The 
outout would be mostly cersonal imoressions In 
the minds of the users of the system under 


Semomstration. 


a Mecaessnent experiments. These are experiments 


where trials are conducted over a wide range Ot 
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mem iitions with little control over sources of 
error. [nem adeaolmersut >= obtain mn idea of ~ Rob 
vel) the System oerforms. The output 1S 
Sucyective opinions of the exoerimenters and 
subjects. [AGA roolmrons en howeworthwnile the 
System 1s and any potential uses are recorded as 


experimental data. 


ie Evaluation exceriments. Desa mare ut ie | mest 
meTaorous tyoe at exceriments. Exnerimental 
moma1itions are carefully controllecd. A number 


0 f reclications are pnerformea LO lowed OY a 


“moeremal analysis of numeric measurement data. 


The assessment exneriment was chosen as the means’ for 
Mmetermining the utilitv of OPINT. No opr1or experiments 
Semeeraina the utility of OPINT kave been performed and 
therefore no baseline currently exists. The results of this 
memerirnent should fill that aao. Subjective wmoressions and 
judarments af C43 users are what 18 needed to helo determine 
the usefulness of OPINT and that 18S what this assessment 
| Re Fee 

The major objective of this exoperiment was to assess 
meemeuti lity of OPINT as a comouterized decision aid in C$ 
BpeOlications. Seconjdary objectives which helred in this 


mea rmcluded tne fol lowinas 


tes To assess tne extent to which OPINT facilitates 


thoroyvoh and timely decision makina. 
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Ee To imvestiqate the user-OPTiT interface and to 
Suagest remedial neasures or Secmnate Wesians 


SeersOoroodriate. 


Ai To Survey user oo0i1NI1ON On treme Oehtt1On. . and 


Dotential usefulness of OPINT in C3 decisions. 


The orimnary oabjective of OPINI is tOumascsise decision 
mawers mM the structurina and analysis of decision problems 
memeyeet al., 1°79, ec. Hes Une eeoulad. molly 9 tf fom stn. Ss that 


meemeret JOeCision or cronuct is delivered tnrouah the use of 


OPINT. lene auestion which is now immediate iLSe "What is 
Better?" [Sar er Cam mot be scenario Gdeoendent or slayer 
deovoendent. ee must be geners) in nature. OPINT's 


perfornmance Tay be be neasured in terms of time (the time 
mreom Oercention of oroblem to decision). A faster decision 
is often times needed and therefare the capability to do so 
Meee termed “good to have". S8ut it is only "agood to have” 
Mmeeeeche Quality of the decision remains at least aS good as 
Met nout the aid. In order to assess this feature, a method 
more measuring the quality of the decision 1s needed. As 


meated earlier, the "correctness" of a decision is extremely 
mitt icuit to measure if at all. In this exoeriment no 
attemot was made to define or measure this attribute. 

what remained was the subjective judgment of tne 
Beojects involved in tne experiment. The subjects were 


Students from the C3 curriculum at the Naval Postgraduate 


BChOo!. C5 students were used Sigee the assessment was 
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memrorred of OPINT's abilitv to helo with C3 tyne oroolems. 
mmese Stucents were all onverational officers with a large 
mmourt Of excerience In ODerational problems in the field. 
They reoresenten the Army, Air Force, Navy, and Marines. 
Seememewere in their last (Sixth) auarter of studies and had 
been exnosed to numerous aecisSion models and aids. The 
muerectS were qrouped into osirs to form a decision team. 
=) | meme subnj=cts had teen trained in the theory and 
orceration of DPINT anda had received at least three hours of 
BeerertiGtion includina one hotir of handsston time. 

Two tyoes of decision situations were used = time 
mmmtiGsa! and tire not critical. lt was understood that the 
range of time oossibilities is infinite and there was no way 
to assess each. Therefore, the two extremes that could be 
encountered were used. 

Two scenarios were followed which reauired the subjects, 
meting as decision nakerss to recommend a snmecific course 
mr action. The two scenarios represented varying comolexi- 
ties which face a C3 decision maker. They were carefully 
desianed to avoid the allegation that they were either OPINT 
mer subject dependent. The first scenario, a Cuodan 
blockade oroblem, was rather complex. it required an 
in-deoth assessment ot intelligence estimates in order to 
predict the occurrence of an uncertain event. In ad@- 
dition, the evaluation of mumerous courses of action was 
mequired. The second scenario wasS an air reconnaissance 


eroblem. nas one was somewhat eas veruas tmer<evsuncer= 
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tain event hat already heen assesset and the decision maker 
mise oniy required to evaluate the courses of action which 
were avarlanle. 4 cooy of each scenario is included in Apne 
Bendix &, 

The subjects olaved one scenario using OPINT and tnen 
olayed the second in a manual mode without the use of OPINT. 
The schedule was varied sa some subjects plaved the dif- 
ficult Cuban croblem first and some the easier reconnais= 
mornce Oroolem. Im Aagaiti0onms the use of OPINT was varied 
sO it was used alternately first or second. Fiaqure ee oorm 
trays a tyocical schedule. Notice the added Gimension. of 
Remmmme critical and time not critical factors. 

It was recoanized the evaluation measures used in the 
exneriTrent would Not be quantitatively measurable. 
Therefore care was taken to carefully select both measures 
Sr effectiveness (MOE) and measures of cserformance (MOP) 
maven Ssupoorted the aoals of the exneriment. 

feeemeasure of effectiveness 1s defined as those 
Mrmeiation reasures which indicate the contribution of OPINT 
to the overall adecus 1onm process (ACCAT Qoerational 
myeruation Task, 1978, o. 6). Measures of vperformance are 
defined as those evaluation measures which indicate a IJevel 
o f mechnical oerformance relative to the internal 
memetioning of OPINT Veseieic Both are aoclicahle to 
decision aid experinents. The MOE's and “OP'S were aesianed 


eo as to: 


lee Measure the contribution o f OPINT to the 
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TIME NOT CREPLCAL 


OF INE MANUAL 


CUBAN ie Rot) a cos ) 
SCENARIO 4 (SECOND) Sa cOND» 


RECON 2 (SECOND) 1 (SECOND) 
SCENARIO 3 (FIRST) a(PiIRrs tT) 





TIME CRITLCAL 


OPintT MANUAL 


CUBAN > (hiked) 6(PIRST) 
SCENARIO 7 (SECOND) 8 (SECOND) 


RECON 6 (SECOND) SCS eCOuD) 
SCENARIO 8 (FIRST) i Geigre Sit.) 





PIGURE 22: EXPERIMENT SCHEDULE 
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decision orocess, not the decision outcome. 


ae Pemobs;ective so they do aot introduce hias. 


Sees Stile, to achieve ecanorny ana ease cf use. 


ae Permit Getermination Gi which  Umct ons wrtnain 
firme Aecisiaon orocess are aided and under what 


Semaitions. 


Di « Yield results in the experiment Pec in are 
Seerendinie to real world Ooerat tone! 


Savironments. 


Ore Be ecerational ly Soi se kere nc imc iit ive ly 
understandable by ootential users (ACCAT 


feerational Evaluation Tasker !1978, on. A-9), 


fee tolilowing is a list of the MOE'S which were chosen 


hor use in the exoertment: 


ie fene eplaosed in arrivina at the decision. This 
includes the time from oercenrtion of a oroblem 
until a decision is reached. COAG On + the 
time elansed in the actual use of the aid will 


be measured. 


2. Tdentification/elirination ot non-essential 


mat oenation. 


3%. Objectivity in determining relaticnsnios among 


factors contributing te fe ipites selection oaf an 


ie 





aitermetive cotrse of action. 


Ds Fisk oerceirvend to ne assoc letcu with the 


mecision-. 


fi 


; mace of use of the decision aid. 


Sie mae Of Nodification of data within the mode). 


Wie Perceived advantaaes/disadvantages Gaines from 


Mmemeanuse of the decision aid. 


Ss Ease of recovery from user errors. 


Data was Gathered on each of the abover throuah the use 
Of Questionnaires. fhe excention was the first MOE. On 
this one, tines were recorded while the subjects were using 
Mmemeeto CaOture the amount of time required to build the 
model, time to evaluate the datar and total Cee 
decision. These times were then compared with times to 
evaluate data and total time for the manual overation. 
Three questionnaires were used to record the opinions of the 
Subjects. A questionnaire was comoleted after the play of 
each scenario and one conoleted after both. Samole 
questionnaires are included as Appendix C. 

Durina the actual Olay of the scenarios, a 
consideration was how to satisfactorily place the subjects 
[arto the mental role required. Thvws Yreltided the ability. of 
the subjects to adeauately assess orobabilities of future 


events and the rearets reauired in the construction at the 
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model. ities ©research desian subsumed this consideration by 
locatinu orobability estimates within the scenarios and 
Berimina the a0als and objectives to be reached. Subjects 
were encourage? ta mrovide their own rearet assessments i f 
they nad exoerierce Tash 1 lot Orob lems. If the subjects 
had no orevious experiencers, an individual was available to 
meres a senior official. PWeySemioOr lett \c1al Ss Pole was to 
orovide rearet data, if needed, and to answer «questions 


momGermina uncertainties or ambiauities. 
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eet eel) Ss Or THE EXPER EMENT 


lommee 86ST aN ficant results from the exoeriment were 
mevealed. Firstr nearly all 7 eeclinilecetce = Stated | t me y 
liked the cadabilities wnich OPINT aave them and would use 
Seeeweerm the future if it was available to them. second, 
metmowaqn they mics met ine —cacatilittvesmot U2 IN?T, all €100%) 
Stated they dic not lixe it ia 12csS present form, 
Borticularly lawoeethme = area of “Wwser=interfaces Finally», a 
proarammina bua Cferror) was discovered in the use of the 
Bemsitivitv analysis. The sensitivity analysis permitted 
the comparison of only tnree courses of actions, Showing the 
BrimeaGtwed regrets for the others as zeros. 

It 1S imoortant to keen each of these results in mind 
as the specific results are read. Many negative comments 
were conditioned oy the statement that if user-interface was 
Better, or if the sensitivity worked correctly, the comments 
would be cositive. The cases where thisS occurs are 
appropriately noted. 

myer Sensitivity analysiS Oroaramming "bug” was found 
during the olay of the Air Reconnaissance Scenario. This 
scenario had five nossible courses of action which needed 
evaluation. GPINT handled the expected value and compvinea 
value calculations poerfectliy. Sut whenever ae sensitivity 
mmaivsis was asked for, OFINTE would not comoute the exnected 


reaqret for all courses oO f act aM. Aadittonal researcn 


76 





revealed that 1m the case of a sensitivity analysis where 
phe ofrobaoilitv of the event was variedes, OPINT would only 
Mmoemomte three courses of action. When the variation of the 
eriterion weiahts was asked for, OPINT would comoute four 
memmses Of action. Any remainina courses of action received 
rearet values of zero. This did not permit a comolete 
evaluation of alternatives and had an adverse effect on the 
meerraads of the subjects toward the decision aid. Ae omen 
Peat haa only peen installed on the PNP 11/70 for one month 
prior to the exvnoeriment, orocramminaq buas such as these 
should have been detected and corrected orior to presenting 
the aid as ocerationally ready. 

All but four subdjects said they felt the scenarios were 
mocmerstic. lthese four felt that priorities established tn 
the scenario were not what they felt they shoula be and that 
prior decisions made aid not seam realistic from their 
viewooint. ivenusce of fhe cealistice scemarios coupled with 
Bmemaorlity of the subjects to inout their own orohabilities 
anid regrets facilitated a decision making environment that 
was as realistic as possitle. 

The eiaht MOE's stated in Chacvter Five were evaluated 
mairpouch the use of questionnaires. A summary of the results 


moe each is snown below: 


1. Time elansed in arriving at the decision. Times 
were recorded in order to capture the time 
subjects were spending on the scenarios, both 


weer and witrout the ue yt Pzot1ion ot OPINT. 
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These times were recorded not to determine if a 
faster decision was oeinag made, hut to Crovigzge 
Pam ac ication of how long it tock to build the 
GeeNtl nodel and evaluate it. In all caseSr more 
tire was soent while using OPINT Enon In the 
manual state. The time reauired to build the 
meet model averaced 45 minutes for the Cuban 
scenario and a minutes for the ee 
meeonnaissance scenario. Tne times were tightly 
arounped as 1S shown by standard ceviations cof 4 
and e@ minutes resoectivelv. The time to 
evaluate THE OPINT model averaged el minutes and 
ld minutes. Aacain small standard deviations of 
3 and 1 minute show tiagntly arouced data. These 
times are very mear to what was expected and 
SNOW 4 aenersal ability of the subjects to build 


ania manioulate the node). 


Identification/elimnination of moM-essent 1a} 
“apormation. Ihe Goal here was to determine if 
the decision orocess was aided by identifying 
non-essential information. 635% of the users of 
OPINT were able to identify some non-essential 
information while only 31% were anle to  in_ the 
manual mode. The items Oi intormation 
identified varied accordina to subject-snecified 
probabilities and rearets. An adaitional 44% of 


the OPINT users stated eroblems with the 
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Sensitivity analysis Ooreventen them from 
possibly identifvina any non-essential 


ma ormation. 


Oojyectivity in deterninina relationshios among 
[memors contributina to the selection of a 
eomrse of action. The goal of this MOE was to 
deternine if the subjectS were aided in the 
Beeson orocess ry determination of which 
mMactors were the most critical. 654 felt OUPINT 
definitely 3a19ed in the decision ocrocess. These 
Subjects definitely liked the way OPINT cromoted 
detailed and nonesubjective analysis of what was 
Mora to haeren and whv. fhesrest found it 
@em1sing due to uncertainty of what the 
Sersolays were actually telling them. 654% also 
peewee a they were able to assess the criticality 
feat ne contrisutina factors when using OPI. 
The evaluator's observation here was that in the 
manual node, one factor was determined to be 
ert ica!l and then the rest iaqnored. Tavs =u 


not occur when OPINT was used. 


Risk oerceived to he associated with the 
Wweeis) on. This MOE was used to measure the 
dearee of uncertainty or certainty in the final 
Gecision. Soot eee subjects in the manual 


mode said thev made the best decision oossiodole 
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while only peo oa the OPINT mode felt that 


eertain. Two tyo2s OF comments were mace . 
yrst; tne Sans it iyi t v oOroblen Wie S aqain 
mentioned aS Aa nha Ggodranc]=. Secon, severa] 


Bueyects felt uneasy about what the numbers in 
the disolays actually meant. For example, in 
the expected value, combined Value, and 
Sensitivity disolayS-e only numoders are shown. 
There 1S no exnglanation of what they mean or 


mire” Numbers are best. 


moece of use of the decision aid. This MOE was 
used to helen determine what tyoe of userwzOPINT 
interface existed. Omly S14. staveget tat. OF INI 
oresented disolays 1n a manner which was easily 
magerstood. SEMSTCIVICY “4nd “commoner value 
aqisolays received the nost comments aS bdeing the 
most Gifficult to understand. Also, there were 
several comments concerning the inputs recuired. 
Many were not sure what was actually being asked 
for and when to hit the carriage return and when 
not toe Almost all 85% stated the disolaysS were 
generated quickly enougqn. No orddlems were 
observed here, even curing times of peak usade. 
Additional comments concerned the use of menus. 


More efficient movement throuah the menu levels 


was wanted. 


80 





mece Of modification of data within the model. 
The qoal nere was to determine how easy eg was 
to eorrect ([eP=1amy Errors = that untrained 
meres’ S will make. &5% stated it was easy to 
make corrections in the model. The only 
nenative comments concerned the use of the menus 
nor this ournose. There existed too much 
merracina through menu selections to make 
corrections in the same area or level of the 


model. 


Ease of recovery from user errors. Tnis MOE 
measured now well the OPINT oroaram recovered 
from iaaoieny PrrORS . Here tne goal was Go 
ceternrine if normal typina or careless errors 
caused major orobnlems. %2% of the subjects who 
made errors felt recovery from these errors was 
BIO T« Comments centered around some simple 
mistakes which caused major oroblems. aie ime 
feta al disolays, a selection of a choice is 
made by movina the cursor to the desired choice 
ang then hittina the 'X!' key for execution. 
However in six caseSr Subjects pressed the 'X' 
key followed by a carriage return. This caused 
the oronrram to halt and the subjects were 
reaguired to deain again. Several subjects felt 
that nebulous inout directions caused several 


errors which coula not be changed until after 
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mae 2mtire model was comoleted. 


Perceived Aevomrages/AaAlrsanvantages aqaained from 
meoeise @¢f OCOPINT., The excolicit consideration of 
a | Pact ors tho rougm the ioromotion of a aood 
thought orocess was the comment most often 
stated. several stated they liked beina forced 
mato. a loaical thought orocess. Also, the 
subjects felt trot nelr lite Rwoceviged them with 
documentation af their decision and would make 
it easier to oresent tne outcome ana justify the 
conclusion. The disadvantaaes listeac for using 
OrFiINi as it currentiy exists centered around the 
user-interface oroblem. There existed a general 
moe< af comfort due to the limited deseriotions 


mmameassistance fron the aid. 


An additional aquest em meoaneceemimas | halt) iG was 
memead Of each subject. They were asked to 
comment on the amount of trainina they felt was 
required for OPINT usaae. The overwhelming 
resoonse was anout two hours of structured 
instruction on the theory and use of the aid, 
followea ay Reme » to =tNe  Mowns ot 9 Paadc=o7 
demonstrations. This amount of instruction was 
exactly what each subject received crior to the 


exoeriment. 
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meee see toes AND. RECOMMENDATIONS 


It was agetermrineda that OP GNI 1S benet ici a fo the 
Marovonm maker in the decision makina orocess. Vinte = ma tora: ¥ 
Bethe SuUGjectsS commentS were either positive in relation to 


meremeor woulct save been with either no program errors or 


Pmmeemeuiser-interface. NFIMNT'’s greatest advantace lies in 
two areas. Eacst > Po MIGrhOTOsess tamecosscinlyeweven forces) 
Sremuiscer Into a soung thouaht orocess. In the heat OT 


memo Or even just daysto-day decisions, it 18s often too 
maeveeto forget the orincivles of aeod decision making and 
mmepemecorscmetnina less. The use of OFINT could help to keeo 
ana t from hanoenina. Secona, “OP lViweeeorovides. “1ts Own 
mocumentation. This documentation is then available to 
oresent aecisions and show why ao oarticular aecision was 
made. et Wamalso-easy to focus on the eritical areas and 
merewaste time on the nonrimnoortant ones. 

[rere are still Some major flaws which need to be 


worked out of the system. Thee 17st RG foremost 1S the 


sensitivity analysis. Program btuaqs such as these can not 
exist in an aid if it is to be used orooerly. Second, 1S 
Maes user-interface rproblem. Tne aid must he designed for 


mae casual lS teary one who. at one time may have been 
Broficient, odut through lack of use has forgotten the key 
requirements. Aaditional corments concerning what 1S needed 


or what is being disolayed are a must. 
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The movement of the user throuah the aia via the menus 


also needs ‘attention. tsers shoulo be saiven the choice of 
meeurmraag to Several ltevels rather than only uo one or down 
qne. 

fem 1?ns!l and by na means the least important 1s eine 
moore m Of Orosram halts. Nothing arieves a user more than 


evga cO start over, esnmecially after a great deal of work 
has already neen done. The ‘svstem “currently displays 
merreteme 1NoCUtsS for verification. Tnere 1S MNO reason why 
Seeemeamerc ic Or Tenu choices could not also be done. 

This author has Bete ail recommendations for both 
ieemovina the decision aid and for future exnmeriments with 
momerses statea earlier, the aid needs to te designed for the 
Betirons! user. This user's knowledce of the aid could range 
trom complete to nothina deoending on amount of usage. The 
aid should ce desianed for tne two extremes and even for the 
Moer in the mitidie. lt méeeds to he both User friendly’ and 
error mole rant ". One techniaue eould See (oot70n4) 
verbosity. For each tisnlay, three sets of comments could 
be precares. The decision maker then selects which set 1s 
desired aenending on need. The knowledgeable user would 
receive only short ohrases which could be keyed on to 
memerate incuts or isolav results. The same tyoe of 
Canabilitv would exist for the other tynes of users. This 
may the wser receives anly what i$ neeced. Another 
variation cf this is to allow eacn user to specify exactly 


what comment3 are needed to Insuce. cerrect uSage and 
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mereroretation. 

The adaition of some sort of "HELP" command would also 
he useful. er eetvecrsion taker aets stuck and needs some 
mmemonmria Or interoretations a simnmle reoquest woula fil] 
muerte ed. User confidence could improve aqreatly with this 
pemole adoaition. 

Since OPIN is redesigned to meet the needs of the 
userSr as stated i179 the results, additional exoeriments 
maummeeeoe conducted to neasure additional capabilities of 
Mme aici. THis exoeriment did not consicger whether a better 
MeeiwsiOn was beinn made. This 1S not an easy problem and 
maermer research and exoerimentation is called for. 

Another area for exnerimentation has to do with the 
area of rearet values . DP Cuserently mas des iamed «to 
ember rearets rather than asins. Some subjects casually 
Startea thev niaht have oreferred to use gains rather than 
rearets. Maaitional exnmeriments to determine if the use of 
agains vice regrets has an impact on user efficiency 15 
Merinitely callen for. 

meme lv, thovoht should he aiven to oproviding a tan Go 
Meeveo0tion to users who have only ae soft copy (CRT) 
@apahility. Further exoerimentation could be done te 
determine if petential users want or even need a Nard cony 
Macnab ility. 

The underlying premise of these results is tnat most of 
the trouble encountered in aetting comouters used in 


decision makina comes from too much emphasis on oits ania 
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Bytes and niniral attention on the user. Trem 66. of 
Beeston Suocort systems such as fPItil, offer an oonortunity 
memes GFOth trivial and immense. ee 1S immense mn tho t 
mie cam 87D) toe computer in decision makina activities 
merece tacage oayoffs could be orovided and trivial Var tot 
meeverrenpresent no mnajyor advance in technology. Suilding 
maoative systens will Fe a difficult and risky venture for 
memeet ime to come. Tne future of decision Suooort systems 
and ooerational decision aits decends on some adjustments 
that hoth the caevelorers ans users must make. But these 
adjustments are smal] anes, angueare: manny catty) tual mat. 
mipeebe mew knowledcae is required. But the oooortunities are 
Pmmesiiestantial, ana tke oersonal and organizational rewards 


hian. 
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APPENDIX A = SAMPLE OPINT SESSION 


em aoint 


imeeemm Arbor 
eemeemy Rhardcony 
3) ADM 3A 
meme m terminal type are you uSina? e 


Please select one of the followina options. 
OPTION 
Pesolay results 
Fdit node] 
Sensitivity 
moad node) 
Create or add to model 
Save node] 
New information 
End session 
meeesired ootion:d 


2 WAN — 


Done MN 


Fnt 
memmormns 8 new model?(v or n)iv 
mmeoenenter 3a title for this model: Lair reconnaissance 


mueermmomortion of the mode) would you like to create? 
OPTICQN 


1 Bemerate influence wdiaqram 
e Generate value/rearet model 
S Mae Baysian indicators 

Y Serrm to main nenu 


Enter desired option:l 


Pemeaeeis the main event of interest? [korean intent } 
Korean intent 


Please enter the outcome lanels for the event you have 
meplied korean intent 


{no action } 
fharassment] 
[shoot sown] 
[ } 


Enter outcore late} 
Enter outcome label 
Enter outcome label 
Enter outcome label 
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Piease enter the orohanilty of each outcone of the 
event vou have labeled korean intent 


no action harassmentshcot vown 

-60 30 10 eeeeeee- -------- 

Nornalize)s values? oO Se O20) 
Tf these values are correct tyoce ao: [ao] 


Which portion of the madel would you like to create? 
OPTION 


1 Generate influence diagram 
e Generate value/rearet model 
5 maa SGavsian indicators 

4 merurn to Tain Nenu 


Enter desired onrtion:e 


Boumewril! be asked to Srovide the criterion to ke used 

im your value judgement, the actions vou ray take, ana 
the associated value/regret matrices for each criterion. 
Please oress return to continue. 


mepeemeet ne mames for each of your criterion: 


faircraftraircrew 
iross of information 
Beet iora! influence 
{recon oroacram 

f 


Briterion 
briterion 
ibriterion 
ler terion 
briterion 


ee ed 


Now enter the labels for each action. 
Bepoamn. (do not fliy 
metoms Lnotgifiec route 
meron: farned escort 
memo m. [hiah oerf a/c 
merpoms (nornal mission 
meron: if 


a ee ee ee | 


You will now be asked to orovide the values for 
maemeotr these value matrices. 
Please oress return to continue. 


Please enter regret/value matrix for 
eameec Poaft ~aircrew 
no action harassmentshoot down 
moa not fiv 00 Of= seer errnre sete te re 
mo mot fiy Deer 0.0 0.0 
If these values are correct tvoe ao: {qo} 
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| 
| 
| 
; 
| 
| 


madified route 0 ©2090 e000 eerere-= 
modified route Uae: -71).0 
ieemese Values are correct tyne ao? 
armed escort 0 2-15 -=$5-2--e-+-- 
armed escort 00 =) 
Pf these values are correct type go: 
high oerf a/c 0 -5 2-15 career e-- 
marc oertf a/c O80) -5.0 
If these values are correct tvoe go: 
normal mission 0 -25 “+100 ere-ee-- 
aorta missicn 7.0 “25.0 
TY these values are correct tyoe ao: 


“100.0 
{a0] 


Please enter rearet/value natrix for 
Current information 
mo action harassmentshoot aown 

geo mot fiy -1090 -100 #100222 cence ceee 
foemet fly = a0, -100.9 -100.9 
If these values are correct tyne go: [aol] 
Heewnri2) route meee 
modified route - 30.0 - 30.0 -3U.0 
Tf these values are correct type ago: [ao] 
armed escort — oe sea SS eee 
armed escort aa) “5.0 -5.0 
If these values are correct type ago: [go] 
momeoerf a/c sy aie) em Oe ee ee 
mame oerf a/c ce -50.0 -S0.0 
Mmiesemes=s values are correct tyne ao: [ao] 
normal nission GRO, = OO SaaS =e eS = 
normal mission Oe (ee0 = 1 0 Oe.) 
Meoeemese values are correct tyne qo: lao] 

Please enter regret/value matrix for 


national influence 


no action harassmentshoot down 


mo mot fiy “50 -100 -/70er+-++- 
mo not fiv -“S0.0 Ore) 
[eet nese values are correct type ao: 
modified route “20 -20 <-S0ee+se+-- 
modified route “20.0 -20.9 
If these values are correct tvoe qo: 


a9 


-=4$90 90 QO 
=e 0 O20 


mired escort 
mermed escort 


[If these values are correct type ao: 
Miah nerf a/c 2-5 -10 -|]0-<-<-- 
high cerf a/c = 60 “19.0 
Pf these values are correct tyoe ao: 
normal mission 0 2-25 -+]00<-+----- 
normal mission Oa “25.9 
mreecurmese Valties are correct tvyoe an: 


Plieas2 enter 
FeEconnaissance oron 
No action 


moomot fly “1090 2-100 -#100-+--2 
Bor mot fiv malty.) = Lowe 0 
mete se values are correct tvyoe ao: 
mowed route -79 -50 -+S0s<-+<-- 
modified route =f). 0 “50.0 
If these values are correct tvpe ao: 
armed escort 9 0 Qee  eeercrrcee 
armed escort ee (eer 0) 
Mmmmemese Values are correct tyoe go; 
Miah oerf a/c -10 #10 -l0Q--+---- 
eran perf a/c -10.0 -10.0 
Iteethes® values are correct tyoe ao: 
momma! n1ission -{10 #70 +80 e+-e--- 
mormal mission “10.0 -70.0 
[memenese Values are correct tyoe ao: 


Please enter a set of 
aircraftwaircrew 
miprent information 
national influence 
reconnaissance poroa 
mts: 100 10 $0 30---- 
Norms: Boe See ll eS 
Tf these values are correct 


ley eno 
type ao. 


Which ovortion of the nodel would you 


OPTION 
l Generate influence diaaram 
2 Generate value/rearet model 
5 Add Bavysian indicarors 
4 Return to main menu 


meter desired ooption:s 


oo 


recret/value matrix 


Narassmentshoot 


Or 


Jown 


INOOGtanNce Werdnt owt Ohetrese criterion. 


{go} 


like to create? 





eease Select one of the following aotions. 
Sey Tor! 


1 Oso lay results 

C Bort node | 

4 Sensitivity 

4 mood node] 

= mmeate or add to model 
5 Save mode] 

7 New information 

a Eng session 


Enter desired ontion:! 


The following disolays are available 
meeTLoNn 


1 Exnmected value 

e Combined value 

4 Event likelihood 

4 Values 

5 Value weiakts 

6 Influence diacram 

7 Return to main menu 


Bermber cesired oaotion: 1 


Expected value/rearet 


Bloemot fiy - 35.4 
modifier route =e Si 
armed escort mie 
hiaqh cerf a/c -9.& 
nornal nission -29.1 


MeerereenoressS rettrn to continue. 


Memo Oning disolays are available 
NPTION 

Expected value 

Comoined value 

event likelihood 

Values 

Value weights 

Influence diagram 

Return to main menu 

Enter desired ootionie 


NOU EWN 


Combined Value 
nO action harassmentsnhnoot down 


me not fiy -32.4 = Ame =f 67 
modified route -17.6 -25.9 “S47 
armed escort -7.4 aa -2-0.9 
eran perf a/c =o “9.4 =o 
normal mission -1.2 =Si5 = Ol, > 


Please oress return to continue. 
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mere following disolayvs are availaole 


OPTION 
] meoectes values 
Ee Bembined valve 
5 Event lixeljynood 
4 Values 
> Value weiahts 
O Influence diaaram 
7 Berurn to main menu 


emcer desired ootion:7 


Please select one of the following ooticns. 
OPTION 

Wrsolay results 

melt model 

Bemsitivity 

load todel 

Beeate or add to model 
Save mnortel 

New information 

End session 

meter desired ootion: 3 


mono MN EF tw ty - 


select one of the followina: 


NPTION 
1 Mmevemt mrocability thresholds 
2 Value weiaht thresholds 
e Return to main menu 


mmyter desired ootion: 1 


meme vent outcome likelihood should be varien? 
OPTION 


1 mo action 
° harassment 
5 BemioOot cmown 


memter desired ootion:!1 


Fxpected Value when 
PrOOooTli ty of Mo vact lon 12S. 


Cate 20 SO os) Ss 0 ee Oe 0S ee 


mo mot fiy -39 =38 =-37 -37 -36 -35 =-34 =-34 -33 =-32e =31 
modified route 732 =-31 2-29 227 2-26 2-24 2-253 221 2-20 7-18 -17 
armed escort “11 -11 tLDk-10% #=9% =9x -B8k& =8t w@= Jk = 7 -6 


high oerf a/c 10 K=10%-10e-104210 $10 “10 “10 29 #9 #9 
normal mission w-4Ge -38 2-34 =30 2-26 22e -18% 2-14 2-10 rox =-2% 
Please press rettrn to continue. 


melect one of tne followina: 


mPTTQN 
l Event orohability thresholds 
c Value weight thresholds 
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3 Return toc main meny 


Enter desired ontion: 3 


Pile 
OFT 


Qn 


met Tone l 
Sernsitivity 
moad note | 


Save nodel 


SNOW LWW oD 


End session 
Enter desired ovotion:6 


mpeiceeselect one of the following 


pe TiON 


mat nocie | 
Bensitivity 
moad node | 
Create or ada 
Save nodel 


BnNOwWNeWNM 


End session 
Enter desjred option: 


Seasa lay results 


Mew information 


fesolay results 


Mew imformatrion 


Create or add to nodel 


mode | 


WES 


meerenrect one of the followina options. 


Got ons. 





Sie Oak c= SAMPLE SCENAP 105 


Cie SUCK oDE SCENAR IG 


Memes “arch 1980, photoaranhs received from military 
murvertilance aircraft showed the existence oO f new 
Mmemerection along the southerh coast of Cuba. Intell agepce 
analysts believe the construction to be anew major Soviet 
nuclear sunmarine sunoort nase. However, they are undecided 
as to whether it will he casable of resunolying only nuclear 
mesmo r botn nuclear fuel and nuclear SLRY missiles. 

Peet ional intelliaence from other sources has revealed 
the cresence of hundreds of new tecnnicians in the area. In 
Fesconse to Anrerican inauiriesr the Soviets have stated that 
Fhey are merely neloinag the Rea build a new merchant 
Bort. They claim the technicians will leave as soon as the 
Bort 1S comoleted. 

Most US ooservers are hiahly concerned over these New 
develooments. Trey feel the oresense of 3a new oortr, cananle 
mmemiie lear retrofit for Soviet submarines, has a serious 
impact on US national security. ‘lo longer would the Soviet 
submarines be recuired to return to home ports. They could 
conceivably oatrol near our shores for unlimited periods of 
Mire. If a nuclear svar ever did break out, a missile 
resupoly base so nearmnvy could be devastating. 

Analysts pvelieve the new construction is caused by the 
possibility of noneratification of the new SALT Treaty. 


They feel if the Treaty is not ratified, (hemecoulcts =wi || 
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hav® a Jumo 17 arms escalation and the US will have no oower 
Mem interfere, Ime analysts also believe that 1f the Treaty 
Memeoeeytied, tne construction will auickly cease. 

Pome TOrTNning of ¢e/7 March, three Soviet shios were 
Merected sailing towards Cuta. Reconnaissance has revealed 
one to be a freianters one an armed escort destrover, and 
the last Ba eSGort suodmarine. Intelliqence sources in the 
movUret Inion have said the freighter 1s carryina the first 
muoely of nuclear fue] and nay oossibly be also carrying 
BeeB™“'s. 

Fears that the Soviets may be trying to quickly 
establish a stronghold in Cuba promotes the Fresident to 
Meager a tyree ship Task Force to intercept the Soviet 
freighter. The mission of the Task Force is to determine if 
mmemeare stioolies of anv type are on boards ana if sor to 
emer :t awaitina further instructions. 

The Soviets immediately resnoonded by statina they would 
Mmereitely deny anv boarding of the freighter and any such 
memo n would be net by military retaliation. Intelligence 
has also revealed the ordering of at least ten Soviet 
submarines into the area and the rapid auamentation of its 
Mmiamtic Force. 

The President has decided he can not allow nuclear 
SupDlies to reach Cura and he feels a blockade of some tyoe 
must be olaced around Cuha. He immediately olaced the 
Atlantic Fleet on alert and ordered them to begin steaming 


Powaras Cuba. In addition, he also olaced the Mediterranean 
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Fleet 9n alert. 


Memeeetyoe and size of blockade has not vet been 
metermined. The President has asked you for your 
recommrennation. The Prasident Seee tically wants a 
meceomnmencation as to whether a total olockade of a]! 


Peemeionor a TNOd1fied blockane of some sort allowing merchant 
Mee cmcarsryina non=-mititary caran to pasSr orf a roving 
Batro! which Hoult. omy detayna SiptOS eeGare yima om) | te any 
hardware. 

Ooerations oersonnel have stated it would require the 
meme otlantic Fleet anda half of the Mediterranean Fleet to 
invoke a total hlockade. A modified blockade would require 
emma Atlantic Fleet while the rovina satro!) would only 
meed half of the Atlantic Fleet resources. 

Intelliaence analysts are unsure what the Soviet 
Paeemtnons will be concernina the situation. They believe 


mauemreootions to be one of three: 
es An attack of tne blockaging forces. 


P< Maat temot to run throuah the blockade waechout 


iInitiatina any aaaressions or 


3. Submission to the blockade. This means the 
Soviets may relent te having sone shios searched 


but may have others return directly home. 


They aaree, however, that twa upcoming events wil) 


helo to determine their motives. The imopendina vote on 
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mye onrl!t freatv and final determination on exactly 
wnat tyce of construction 18 Felng cone on the coast are the 
kev meaues 1M assessina their intentions. lay acon too y 
they fee] Anes com el reaty vote will pe Seftected Fy Bath US 
Seieeere ss COIAION concernina the fTreatv and current world 
Rormiom concerning the situation. lreeUN@= Security counc i | 
is scheduled to meet in two days to discuss the situation. 

femamalysts have assessed the pronatility of US public 
meee betra oro-SALT as 70% and anti-~SALT as being 30%. 
mano 1nioOn is assessed as having a 40% chance of being 
meen 504 oro-Soviets ant 30% of wveina neutral. The 
mmemeysts also currentiv aqree to a 609% Jlikelthooa of the 
construction beina to suvoovort full nuclear resunoly, 30% for 
mrmiy Naval resunonly (non-armaments), AMG aL Oeo aS. sonnei 
mommercial oort. 

The jtems tne President wants considered In tne 
M@tetnon of a best course of action are (Cin order of 


mmOortance): 


1) National Security = Tne oresident feels he can 


not allow the Soviets to establish a susply base 


me Cunha . 
2) Safety of US Nationals currently in Cuba. The 
WS has recently beaun exnortina numerous 


commercial oroducts to Cuba in an effort Eo 
Boost a sasaqing economy. At present, It 15 


estinated there are at least 2000 Americans a’ 


a 





eS) 


4) 


2) 


mee a. 


meoeotmo | 1c OO 1NION. This 1S an election year and 
analysts feel there will be nublic outcry if we 
merce the Soviets into a military confrontations 
but tne Jeuplyve ows) be impressed by strong 
measures which force the Soviets into accepting 


sue cemands. 


Puioan reaction. The US has only recently begun 
trade with Cuba anc is anxious to continue this 
if at all oossible. Any tyoe of blockade 
immposed around Cuba will have a serious impact 


on these trade agreements. 


Norld opinion. Any confrontation between the US 
eee USSR 1s undesirable. If the SS is 
successful with the blockade, world opninion will 
me high. However, if we orovoke aaaression by 
SiGe actionss we wi1il lose addytional worla 
Suoport which is deseerately veing sought by the 


PResiqdent. 
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ALR RECONNAISSANCE SCENAPIO 


Nm 9% “arcn 1989, an unarmed Mohawk reconnaissance 


memermartc avorted its nission over Korea. Lieereturmed to ts 


nase where tre crew reo0orted an interception by a 
meme r=tyoe aircraft from North Korea. The crew of the 
unarmed Yonawk had not Stayed around lonq enouah to 


determine whether the intercepter was there to harass, 
shoot, ifoemtify, or acd hust hecoened salona ewhiile on 5 
Boerne trainina fliaht. Another reconnaissance flignt is 
scheduled feet tororrow ang the commander needs a 
meeemrendation concerning future fliants. 

Peerer furtner discussion, the commander desires the 
followine ootions evaluated in light of what has haooened 
and what Maem te)lliaence staff conclude are North “orea’ s 


mmr entions: 


is ioemot fiv into the area anymore. 


Ce mveeimto the area on a modified route. nwes 
meat 1e0 route would be less sensitive bout also 


Wess oroductive im collection ootential. 


Bis Fiy Tale} the same area but now with an armed 


escort. 


a. aly int © the area with hiagh-oerformnance 
marc ratt. The aircraft would be less vulnerable 
but would not collect data as well as the norms } 


Baamerart. 
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5S. 06CUWFLULYhClUlA8CONOr Ma) missions do not chanae anything 


mmen Orevious flights. 


ieee coT™mander is mostly concerned with the possible 
Meoemor the aircraft and crew durina the mission. However, 
mememroa'SO concerned with the loss of prestige or politica! 
momrecemce if fliahts were stooned or altered. [he loss” of 
merrermnmation ana the sossihie tmpact n the reconnaissance 
meeermanm in other areas are also important factors. 

The intelliaence section believes it has assessed the 
possihle intentions of the North korean Government on future 
flights. 8ased on oast situations and current capabilities, 
maemweetree)] there is only a slim chance (104) the Koreans wil! 
mevemecm shoot future aircraft down, a moderate chance (30%) 


meee) ) Aarass, but orobably will do nothina (602). 
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aPPFEMNDETX C = SAMPLE QUESTICNNAIRES 


Bee ONNC TPE FOP JTFAM MEMRERS wITROUT OPINT 
OME : 

RANK: 

mast e 

BCENART(): 


WHAT NAS YOUR RECOMMENDED ACTION? 


Semen YUN RECUMMERMD THAT ACTION? 


Beets THE FOLLONING QUESTIONS AS INDICATED. 


mumeremmyviou feel this scenario was realistic? lf nor olease 
Semment in the soace oroavided. 


ae Yes 

Ge. ‘lO 
@. viere you aole to idantify any non-essential information 
Stiemmee your evaluation of the courses of action? esr 
mmeeewas that information? 

a. Yes 

ie NO 
Mere You able to assess the criticality of the factors 
moyen contributen to the choice of the recommended course 


of action? If yes, which factors were the most critical? 


ae Yes 


O. No 


4. Do you feel confident that your recommendation was the 
"hast", based on the information orovided? If nor why not? 


a. Yes 


ye be 


Pon 





Beers TRE FUR TEAM “MEMBEPS ITH OPINT 
NAME: 
Rath 3 
eave 2 
BeEUARTO: 


WHAT “WAS YOUR RECOMMENDED ACTIGN? 


See YOU) PECOVMEND THAT ACTION? 


Moe oCR THE FOLLOWING GUESTIONS AS INDICATED. 


memmermervou fee}] this scenario was realistic? Lt no. oe ase 
@emment ifn the snace orovided. 


Qe Yes 

er NO 
Mme Yo; anole to identify any non-essential information 
Sormma your evaluation of the courses of action? Jf yes, 
memes tCHat inforration? 

Ae Yes 

On, No 
- Memenyou able to assess the criticality of the factors 
mmenecontridouted to the choice of the recommended course 
mi action? [meee S, which tactors werent noemoctmernltic¢cal 2 

as Yes 

0 e ‘| Oo 
4“, Do vou feel confident that your recommendation was the 
Mhest", hased on the information provided? If nor why not? 

Be Yes 


oe) 0 
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peeeoad OPINI oresent 
was easily understood? 
momeeimaerstand? 


ats Yes 
D m ie) 
Seer OPINT disclay its 


momen disolavs were stow? 


Bae Yes 
er 1O 
feomas it easy t7 
Gemment On your orohlems. 
ae Yes 
wee NO 
mead OPT recover wel} 
made? If nots what was the 
Ge yes 
oo) |6=OCLO 
or “i/ - 


the evaluated data 
tt nos 


moawe corrections 


fimo me aicy 
result ¢ 


nS 


which oisolavs were 


EA stines Guiiek!) y enough? 


iit he | mogde |! 2 


Ieee GemMake omy errors. 


Paes nT oanee Gew hen 


Atta c ak 


Leia velice 


If et, 


errors you miaht have 
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Dee TeeE FOr FEAM MEMBEPS AFTER COMPLETING BOTH SCENARIOS 
NAME: 

RANK s 

eA st 3 

meee iowER THE FULLOWING QUESTIONS AS INDICATED. 


meena: extent did OPINT aid in your assessment of 
what event was aoing to haonen? 


a. Confused 

oo poe ne lp 

eum einforcea own ideas 

meee Clarified 

eee nl iantened 

ff. Had no need to assess an event 
meeominngt extent did OPINT aid in your aecision process 
momen -ctinag 2 course of action? 

ae Tanored 

Ds Relied upon it 

Gee Added confidence 

fee -iasS Confusing 
See o you think tnat OPINT heloed to sneed up the decision 
making crocess? 

a. Yes 

| NO 
Bee o> you think vou would have ocertormed better with or 
Pepemout OPINT? 

ae tote 


Ds rer thout 


1Q4 


a - 7 7 


‘ 
ao 1 @ | «ur if y 7 ¢ fh Seriad ‘ 
: ra 


- 


oi 7 





Sie Mmemevou fee) that CPINT could be imnroved uoon? la MES 
poewetoch areas? 


[momo feo)| Gd2=cision makers shoulda use CPINT+-r or shoula 
momeeme tt to rechnical exoerts? 


ae Mecision rakers 


‘ene imeennical exoerts 


[emeenrero VOU Cerceive as beina the aqreatest advantace in 
wotwme oP iNT? 


Se What ‘Io you rerceive as beina the areatest disadvantaae 
foeeceiaag OP TNT? 


eons ye) use OPINT in the future if tit was made 
eemmremle tO vou’ Please comment. 
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4 son 4 
sae 0S : ee 


‘ove sand 


an ‘eoinag 
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